Why New Film Sections Never Show Up in Google: Not Crawled, Not Indexed, or Indexed but Invisible
A film desk's new interview, festival or review section pages live, but Google does not show them. How to separate not crawled from crawled-but-not-indexed from indexed-without-impressions, which robots rules, sitemap files and internal links you can verify yourself, how sitemap lastmod should be maintained when pages are added, rewritten or merged, and what a URL-scale indexing service does and does not promise.
Brand promotion: this article introduces Guangsuan's Google indexing services. It is a vendor introduction, not an independent review or a commissioned evaluation.
A film desk publishes a new interview series on a Tuesday. By Friday the section page opens, the navigation link works, and a reader can paste the address into a message and get the piece. Two weeks later, a search for the series title returns the homepage and nothing else. Nothing is broken. The page simply is not showing up, and not showing up is a symptom that covers at least three different problems, with three different fixes and three different answers about who should do the work.
For an editor the tempting move is to treat that as one problem and go looking for one remedy. It is cheaper to split it first. This guide walks through how to tell not crawled from crawled-but-not-indexed from indexed-without-impressions; which robots rules, sitemap files and internal links you can verify yourself before anyone quotes a price; how a sitemap's update rhythm and lastmod values should be maintained when pages are added, rewritten or merged; what to read when the Search Console report and what you actually see disagree; and what a URL-scale indexing service is paid to do, how it is verified, and where it stops.
Three states that all arrive under the same complaint
Direct answer: discovery, crawling, indexing and appearing in results are four separate stages, and the label a tool gives you usually collapses the middle two. Guangsuan's own GSI description draws the same line, splitting its flow into discovering the URL, crawling the page and leaving the indexing decision to Google, and stating that whether a URL is indexed, when, and whether it stays indexed remains Google's call.
| What you are seeing | What it establishes | Where to look first |
|---|---|---|
| Google knows the URL but has not fetched it | Discovery failed. Guangsuan's Chinese-language service page calls this state 已发现,尚未抓取 (discovered, not yet crawled) and names site links, server stability and page scale as the discovery-side items worth reviewing. | Whether the sitemap you serve actually lists that address, whether an ordinary internal link reaches it, and whether the host answers reliably. |
| Google fetched the page and did not index it | Crawling worked, the decision to keep the page did not. Google's Page indexing report documentation records this as a state in which the page was crawled by Google but not indexed. | The page itself: duplicate versions, the canonical Google selected, and whether the page is indexable at all. Guangsuan's published scope says the service cannot bypass Google's quality judgement or replace content and technical repairs. |
| Indexed, but no impressions for the query | The index state is settled. What is left is relevance and competition, not crawl access. | Whether the title, opening paragraph and headings answer the question a reader actually types, and whether a section archive already covers the same intent. |
None of those labels is a verdict on its own. Crawled but not indexed is not proof that the writing is thin, and discovered is not proof that you need to pay for crawl help. Read the label as a pointer to the next check, and treat it as one URL's question rather than a statement about the whole site.
What you can verify yourself before anyone quotes a price
None of the steps below needs a developer, a plugin or a budget, and each one removes a possible cause. Work through them in order rather than jumping to the expensive one.
- Fetch the exact address while signed out. Open the final URL in a private window, not an editor preview and not a shortened link. Record the final address after any redirect, and confirm that the page text itself comes back in the delivered HTML rather than appearing only after a script runs. A page that returns a success status with a login wall, a challenge screen or an empty template has not been delivered to Google either.
- Read the robots rule that matches that path. robots.txt controls crawling, while a noindex directive asks that a crawlable page be left out of results. Google does not index pages blocked by a robots.txt rule or a noindex tag, and Google can index any URL it finds unless that page carries a noindex directive. So read the rule that applies to the specific path and the user agent involved, not the file as a whole. A rule aimed at an admin area or a preview directory is often harmless, and a rule written for one section can quietly catch another.
- Find out who is outputting noindex. On WordPress, the site-wide switch lives under Settings, then Reading, and the per-post and post-type settings are separate switches in the SEO plugin. After a rebuild, the suspects worth checking are a staging-era setting that was never turned off, a post type excluded from indexing, and a template that still carries the directive. Compare a new section page with an older one that is indexed: if several pages share the same noindex or the same fetch error, look at the shared setting first; if one URL is odd on its own, look at that page's content, canonical and configuration.
- Check the sitemap that is actually in effect. Do not assume the file you submitted is the file being served. A theme, a plugin and an old static file can each maintain a different version, so open the exact address you submitted in a signed-out browser and confirm it returns XML rather than a login page or an error page served with a friendly status. Then check the contents, not only the status: it should list the final canonical addresses you want in search, with absolute URLs whose protocol, host, case and trailing slash match the live pages, and without redirects, 404s, duplicate variants or noindexed pages left in as normal targets. Keep one current file, because a second older file still advertised by a plugin or by robots.txt splits the signal. One uncompressed file should stay within 50 MB and 50,000 URLs.
- Confirm the internal path and the canonical. Every page you want found should have at least one ordinary crawlable link pointing at it from elsewhere on the site. A sitemap entry on its own is a weaker path in. Compare the canonical you declared with the one Google selected, point your internal links at that same canonical URL, and use canonical for duplicate or near-duplicate content only. Sitemap inclusion is a weaker signal than redirects and canonical, and none of the three forces Google to adopt a version. Pointing a unique section page, such as a festival archive or a themed column, at the homepage tells Google that the page you care about is a copy of the home page.
- Write down the before state. One row per important page: URL, page type, final status after redirects, declared canonical, the robots rule for that path, any meta robots tag or X-Robots-Tag header, the Search Console status and last crawl date, and the date on which you read them. Without that record, the only later comparison available is an impression that things look better.
In short: fetch each address signed out, and read the robots rule for that path, the source of any noindex, the canonical and the internal link that points at it. Those checks, done against a signed-out fetch and the sitemap file you actually serve, are what separate a crawl problem from a page problem.
That last row matters more than it sounds. A paid service can only work with the list it is given, and the same list is what you read afterwards to judge whether anything changed.
How often the sitemap should change, and what lastmod should say
Guangsuan's Chinese-language guide on sitemap update frequency puts it this way: the sitemap should be updated when pages are published, substantively modified or their address status changes, and there is no fixed once-a-day rhythm that suits every site. Its practical point is that the list must agree with the live pages, and that you cannot refresh the date of pages you did not touch.
Three clocks use the same words, so keep them apart:
- The page change. A lastmod value should reflect that page's most recent substantive modification. Guangsuan's guidance treats a change to the main copy, to structured data or to important links as a reason to update the date, while an unrelated footer edit is not. If your CMS cannot record a reliable per-page modification time, treat the date as unknown rather than writing today's date on every URL.
- The file generation. A sitemap file may be rebuilt because one new interview was published, and that does not mean every page inside it changed. If your generator writes the current timestamp into every entry, the field carries no information and you should stop reading it as one.
- Google's processing. File generation speed is not Google reading speed, and neither is indexing. Google's own documentation says a sitemap helps search engines discover URLs but does not guarantee that everything listed will be crawled and indexed, and Guangsuan's own note is blunter: a successful sitemap submission only means the submission step was completed, not that products and articles are indexed.
For a publishing site the change list is short and mostly mechanical:
| What changed | What the sitemap should do | What to check alongside |
|---|---|---|
| A new section, column or series page is published | Add the final canonical URL | The page is publicly reachable, carries no noindex, and is reachable from navigation or the archive |
| A review's body, credits or verdict are corrected | Keep the URL, and update lastmod only if the change is substantive | The live page and the cached version agree |
| A festival tag archive is merged into one page | List the address you intend to keep and leave the retired one out | The old address redirects properly and internal links point to the page you kept |
| A page is deliberately retired | Remove it from the list | Whether it should return a 404 or 410, or redirect to a real replacement |
Two habits waste a week. Resubmitting a sitemap that has not changed repairs nothing, and asking Google to recrawl the same unchanged URL does not make it crawl faster. Guangsuan's sitemap guidance makes the same point about the old ping endpoint, which has been discontinued: the way to hand over a sitemap address is robots.txt or Search Console. Google does not use changefreq and priority, so setting changefreq to daily buys nothing.
Reference: Google Search Central, Learn about sitemaps.
When the Search Console report and the live results disagree
The Page indexing report shows the indexing status of the URLs Google knows about in your property. The Performance report shows what happened in search for the queries and pages you filtered on. Neither is a live check of whether one page appears for one search, and a site: query is not an inventory: Guangsuan's own service page states that a site: query does not necessarily list every indexed URL, and that where you have property access, the URL inspection tool and the page indexing report are the better evidence. The same page is careful about its own illustrations, noting that its site: query screenshots are not GSC reports and do not represent current results, a complete index count or a promised outcome.
When the two disagree, work through the order below. The Page indexing report documentation in Google Search Console Help is the reference for the status labels used here.
- Confirm the scope first. A Domain property and a URL-prefix property do not include the same URLs, because protocol, host and path change inclusion. Record the property type and the filters you used, then compare the unfiltered report with the page, query, country and device view you had in mind. Check the report's update date and its completeness notice before drawing a conclusion. Google's documentation notes that when you add new content it can take a few days for Google to index it, and that for a new page or site it can take about a week before crawling and indexing begin, which is a reason to wait rather than a diagnosis.
- Read the per-URL label literally. URL inspection shows the version Google knows, when it last crawled the address and which canonical it selected. A live test shows what Google can fetch right now. Those are two observations with two dates, and keeping them apart stops a stored record from being mistaken for the page as it stands today. A URL Google has not crawled points at the path into the page; a URL it crawled and did not index points at the page itself.
- Ask whether the page answers the query at all. If the URL is indexed and produces no impressions for the query you care about, the remaining question is relevance and competition, not crawl access. Read the title, the opening paragraph and the headings against the wording a reader would type into a search box, and check whether another page on the site already covers that intent.
- Do not turn one label into a site-wide rate. Google's guidance is to aim for the canonical version of important pages rather than to expect every URL on a site to be indexed, and its documentation notes that most sites carry some unindexed pages while the important pages should be indexed. So a grey row is a question about one URL, not a measurement of your publication's health. The same caution applies to the Performance report: impressions count appearances your own pages received for queries, which is not the same as how often the topic is searched overall.
One nuance worth keeping: Google's documentation notes that Google can index any URL it finds unless that page carries a noindex directive, so a path blocked in robots.txt is not the same thing as a page permanently removed from results. Read the URL's own state instead of assuming either outcome.
Reference: Google Search Console Help, Page indexing report.
What a URL-scale indexing service is paid to do
Guangsuan's GSI, its own name for a Google indexing promotion service (Google Speed Indexing), is described by Guangsuan as not a Google product, not an official partnership and not a priority-indexing permission. That boundary is worth reading before a call, because it decides which of your three states it can address at all.
It can address the first state. Guangsuan describes the work as operating around the URLs you submit, supported by its own GPC crawler pool and related link structures, to encourage discovery and crawling.
It does not touch the other two. Guangsuan's published scope states that the technical service solves the discovery and crawl opportunity, while whether a URL is indexed, when, and whether it stays indexed is decided by Google, and that the service cannot bypass Google's quality judgement or replace content and technical repairs. Its own pre-submission condition follows from that: URLs must be normally reachable, and pages that are crawl-blocked, carry a noindex, redirect or duplicate other content should be checked first.
Scoped by URL volume, not by expected traffic. The published tiers run from a batch of 1 to 100 URLs, through 1 to 1,000, to 10,000, to 100,000 and above, each with its own published price and reference window. The number you submit is what sets the scope, so a desk submitting a few hundred article URLs is in a different engagement from a catalogue of ten thousand.
The published periods are references. Guangsuan states that the published cycles are service references, affected by page quality, site state and Google's processing, and that no fixed number of days is promised. Read a quoted window as a published range, not as a delivery date that belongs in your contract. The reference indexing-rate interval published alongside those tiers is described in the same place as a reference rather than a guarantee for any batch of URLs.
Verification is agreed in advance. Guangsuan states that the service scope, the observation period, the verification method and the reporting are confirmed before cooperation. If nobody on your side holds Search Console access, settle before the work starts how results will be checked, because a third party without property access cannot read the URL inspection tool or the page indexing report.
What to bring to that conversation is the list you built earlier. Guangsuan's own guidance for sitemap enquiries asks for something close to it: the sitemap address, the main page types, the number of URLs expected, a sample of representative GSC statuses and the inspection date, plus a note of what has already been fixed so that work still needing a developer is not confused with work suited to crawl support.
Only when pages are reachable, indexable and linked, and Google still has not fetched them, is crawl promotion a sensible purchase, and then judge it on the state change you recorded rather than on an activity report.
A short decision order for a film desk
- Record the symptom per URL, not per site: sample a review, an interview page, a festival dispatch and an archive page.
- Fetch every address signed out, then read the robots rule for that path, the source of any noindex, the canonical and the internal link that points at it.
- Open the sitemap that is actually served, check its contents rather than its status, and keep lastmod honest.
- Read the Search Console label literally, and keep index state and visibility apart.
- Treat crawl promotion as the last step, after the checks above are clean, and evaluate it on the state change you recorded rather than on an activity report.
A hypothetical case, used only to show how the order works and not a client site: a new festival archive page is published, linked from the navigation and listed in the sitemap, and Search Console shows it as discovered but not crawled. Signed out, the page returns its own text with a self-referencing canonical and no noindex directive. That is the first state, so promoting discovery and crawling is the sensible next step. Change the same page so its canonical points at the homepage, and the same label now points at a page-level fix instead, which no amount of crawl activity repairs.
Three places on this site are good starting samples for the check, because each is a section page with its own filters and its own audience: the reviews archive, the interviews desk and the festivals desk. Two earlier pieces cover the measurement side of the same work, one on why the Search Console date sometimes lags behind your publishing, and one on which of the nine metrics to read before changing a review page.
Section samples: reviews archive, interviews desk, festivals desk.
Earlier measurement pieces: Search Console lag before rewriting reviews and the nine metrics to read before changing a review page.
Where Guangsuan fits
If your section pages are reachable, indexable and linked, and Search Console still shows that Google has not fetched them, then Guangsuan publishes its own GSI Google indexing service page with the flow, the URL conditions, the volume tiers, the verification route and the stated limits. Read it before a call: Guangsuan's GSI Google indexing service page.
For the maintenance side of the same problem, Guangsuan's guide to sitemap update frequency covers when to update the list, how to handle lastmod, and what to do when pages are added, changed or merged. Both pages are written in Chinese, and both say the same thing this article does: promotion covers discovery and crawling, and Google makes the indexing decision.
Frequently asked questions
Do we need to resubmit the sitemap every week?
No. If the file was read successfully and nothing in it changed, resubmitting repairs nothing. Fix what the per-URL state points at, and let normal recrawling continue.
Our review is indexed but never appears for the film title. Is that still an indexing problem?
Not mainly. Once a URL is indexed, appearing for a query depends on relevance and competition and on how results are assembled for that searcher. Check that the title, opening paragraph and headings answer that question, and that a column or archive page does not already cover the same intent.
Is there an API that will index our pages for us?
Guangsuan's sitemap guidance notes that Google's Indexing API is limited to pages containing JobPosting, or BroadcastEvent inside a VideoObject. Ordinary review, interview and section pages fall outside it, so crawlable internal links, a current sitemap and a recrawl request for a few important URLs remain the documented route. A vendor describing instant indexing for ordinary pages is describing something else.
Does GSI guarantee that a submitted page will be indexed?
No. The service promotes discovery and crawling around the submitted URLs; whether a URL is indexed, when, and for how long it stays indexed is Google's decision. The published periods are service references rather than fixed completion dates.
Can a small editorial team do this part itself?
Most of it, yes. Fetching an address while signed out, reading the robots rule for that path, checking the site's own indexing switch, reading the sitemap you actually serve, and confirming internal links and canonicals are all doable in a browser. What usually needs a developer is a template or plugin change, a robots.txt rewrite, or a server-level fix after a rebuild.
One last note on the commercial side: this article recommends Guangsuan's services, and everything above about what those services cover comes from Guangsuan's own published pages rather than from an independent test. Read the scope, the verification route and the stated limits before you buy.