Infinite Scroll SEO: Keep News Archives Crawlable

Yes, a news archive can use infinite scroll and still be crawled, but only if every slice of the feed also lives at its own stable URL and the slices are joined by ordinary links. Google's documentation says its crawlers do not click buttons and generally do not run JavaScript that waits for a user action. An archive that reveals older stories only after scrolling can therefore look, to a crawler, like a single short page. The order of work is the pattern first, then the HTML that has to exist, then the address-bar behaviour, and finally a test with scripts off.

It is written for developers, technical SEOs and product owners who run a content archive. If you need the general rules for paginated category pages, read our pagination guide for category pages; for images and iframes that load late, see lazy loading and SEO. The mechanics are covered there. This article stays on one decision: scroll, numbered pages, or a mix, for a collection of dated stories.

Why a scrolling feed can hide your older stories

Think about how a crawler discovers a URL. Google's link documentation explains that it generally crawls links found in a elements that carry an href attribute. Links built from other markup are often not extracted. An infinite feed that appends stories through a script, with no href to a story or to the next slice, gives that process nothing to follow.

What counts as an action nobody performs

The JavaScript lazy-loading documentation states that Google Search does not interact with your page, so content that appears only after a scroll or a click is at risk. The pagination documentation says the same thing from another angle: Google's crawlers do not "click" buttons. For an archive this has a blunt consequence. If story number 400 appears only after the twentieth automatic load, no mechanism in the page tells a crawler it exists.

The documents do not publish how far a crawler scrolls, which screen size it uses, or how many loads it will trigger. Do not build a plan on guesses about those details. The reliable design is one that needs none of them.

What is not a problem

Scroll-triggered loading is not forbidden. The same documentation recommends loading content when it enters the viewport, using native lazy loading, the IntersectionObserver API or a viewport-aware library, as long as the content does not depend on a user action to appear. It also warns against lazy loading content users will see immediately, because that can make loading feel slower. So the real question is not "scroll or no scroll" but "can every story be reached without scrolling at all?"

The four patterns, compared for an archive

Google's pagination documentation compares three patterns and states trade-offs for each. Add the date-based archive that many newsrooms use and you have four realistic options.

Decision rule

Use this order of questions:

1. Will readers need to reach stories that are months or years old? If yes, the archive needs numbered or date-based pages, whatever the front-end feels like.
2. Is the feed mostly a "what is new" stream? Then a button or scroll over the newest items is fine, provided the older part sits behind real links.
3. Can your team maintain server-rendered URLs for every slice and update the address bar from the script? If not, choose numbered pages and skip the scroll layer.
4. Does adding stories at the top push older ones down the list? Then plain page numbers will drift, and dates are the safer anchor.

Google's documents describe the pros and cons of each pattern but do not say which one a publisher should pick, so this order is a working rule for archives, not a Google requirement.

Applying the rule by age of stories

Building the crawlable layer under the scroll

The recommended structure for scroll-based lists is paginated infinite scroll: ordinary paginated pages underneath, with a script that stitches them together for the reader. Everything below describes that underlying layer.

One persistent URL per slice

The lazy-loading documentation says to give each chunk its own persistent, unique URL and to keep the content at that URL stable. Its example is an absolute page number such as page=12 in a query string. It warns against relative parameters, like a "yesterday" value, because the content behind such a URL changes over time and is harder to index. For a newsroom, that means a URL such as /archive/economy?page=12 is fine, while /archive/economy?range=last-week is a convenience filter and a poor permanent address.

Plain links from slice to slice

The pagination documentation asks you to include a elements with href from each page to the following one, and to link from the pages of a collection back to the first page. The back-link hints that the first page may be the better landing page. It also states that pages should not use fragment identifiers, the hash part of a URL, for page numbers, because Google ignores fragments and may skip the link.

Canonical, titles and the old rel attributes

Three further points from the same page:

• Do not make page one the canonical URL of the whole sequence. Each page gets its own canonical.
• Titles and descriptions may repeat across pages, but distinct titles are normally recommended. "Economy Archive, Page 12" costs nothing.
• Google no longer uses rel="next" and rel="prev". Other search engines may still read them, but they are not a substitute for working links.

For sort orders and filters, the documentation suggests keeping variations of the same list out of the index, for example with a noindex robots meta tag or a robots.txt rule for particular URL patterns. It does not suggest applying that to the numbered pages themselves.

Keep the address bar honest

Per the lazy-loading documentation, when a new chunk is scrolled in and becomes the main visible content, the script should update the URL with the History API so readers can refresh, share and link to what the browser shows. The documentation does not say whether to use pushState or replaceState; the mechanics are in the lazy-loading guide. For an archive the decision is narrower: which tiers need it at all. The live feed over the newest stories does. Closed monthly archive pages do not, because each one already is its own address.

Slice titles that tell readers where they are

Because repeated titles are allowed but distinct ones are normally recommended, give each slice a label that helps a person. A title such as "Energy Archive, Page 12" plus a description that names the date range of the stories on that page tells a reader, and a crawler, what makes the page different from page 11. This is plain labelling, not a ranking tactic. Avoid stuffing extra terms into the title; Google's link guidance warns that forcing keywords into anchor text is keyword stuffing under its spam policies.

When to drop the scroll layer

Three signs suggest moving to numbered pages. Readers use the archive to find a specific old story and want to jump by date, which a scrolling feed handles badly because it gives no sense of position. Your team cannot commit to maintaining server-rendered slice URLs and History API handling, and a half-finished scroll layer is more fragile than plain pagination. Or the footer holds links you cannot afford to hide. Any one of these is enough to prefer the simpler pattern.

The load more button, done properly

A load more button fails the same test as a scroll event: if it only fires a script, a crawler cannot follow it. For an archive the question is where the button belongs. It suits the newest tier, a short current stream, and it should be written as a real link whose href points to the next archive page, so that older tiers stay reachable. When JavaScript runs, intercept the click, fetch the next slice and append it. When it does not, the link simply navigates.

Anchor text for the control

Google's link documentation allows other attributes, such as onclick or class, next to a valid href. It asks for descriptive anchor text and gives "click here" and "read more" as examples to avoid. A control labelled "Next page of economy stories" says more than "More".

Reaching the footer

This is a usability concern rather than a documented SEO rule, but it affects links: in an endless feed, the footer keeps moving away. If the footer holds your contact details, section links and legal pages, readers may never reach it. A load more button after a set number of slices, or a fixed side column, solves that and keeps those links stable.

Testing: can the archive be reached without scrolling?

Build a repeatable test and run it after each template change. The following checklist works for development and for staging.

Checklist

• Open a deep slice URL directly in a fresh tab. Does the server return that slice's stories in the HTML?
• Disable JavaScript and repeat. Are the story links still present?
• Does every slice link to the next one with a real href, and back to the first?
• Is the canonical on each slice its own URL?
• Are there any hash fragments used as page numbers?
• Does the address bar update while scrolling, and does refreshing keep the reader in the same slice?
• Are sort and filter variations kept out of the index?
• Does every story have at least one link from somewhere other than the infinite feed, such as a topic page, a date page or a related-stories block?

Rendered HTML in Search Console

The lazy-loading documentation recommends the URL Inspection tool in Search Console to confirm that content appears in the rendered HTML. For an archive: inspect a deep slice, run the live test and look in the rendered HTML for the story links and the next-page link. If you use the interface in another language, menu names may differ slightly from the English labels used here.

The documentation does not describe any other testing tool or a crawler viewport size, so treat browser automation as your own addition. Many teams add a small check to their build pipeline that loads each template with scripts off and asserts that the next-page link exists.

Server logs as a second opinion

Access logs show which slice numbers bots actually request. If requests stop at the first few pages, the chain of links is probably weaker than you think. Logs are a diagnostic, not a guarantee. Google's Search Essentials page notes that meeting its requirements does not guarantee that content will be crawled, indexed or served.

Example scenario: a business daily with a decade of coverage

Example scenario: A business daily keeps ten years of reporting under topic pages such as "Energy" or "Retail". The mobile page for each topic is a scrolling feed. Editors notice that older analysis pieces get almost no search visits, even though internal search shows readers still ask for them.

The audit finds two things. The feed loads each batch through an API call, with no per-batch URL, and the topic page's canonical is the same for everything. The desktop version has a "Page 2, 3, 4" strip, but it is hidden on mobile with CSS and not present in the mobile HTML at all. The team makes four changes while keeping the scrolling experience:

• Each batch gets a server-rendered URL with an absolute page number.
• Each page's HTML contains links to the stories and to the next page.
• The script updates the address bar as batches become visible.
• Stories older than two years move to monthly archive pages whose content does not shift.

No number is quoted for the result because it depends on the site's authority, the age of the content and crawl demand. What the team can show is a passing checklist and rendered HTML that contains the links.

Who does what

Drift: why page numbers move and what to do about it

Every new story pushes older items one position down. Page 2 today holds what page 1 held yesterday. That is fine for readers, but it undermines the idea of a stable URL. The documentation asks for persistent URLs and unchanging content; it does not discuss how to handle drift on a live feed.

Options for stable archives

• Date pages: a month or day is closed once it ends, so its content stays put.
• Frozen older slices: keep the live feed to the first handful of pages and move everything behind it to date-based URLs.
• Offsets from a fixed anchor: compute pages from the oldest item backwards, so new stories do not shift old pages. This costs more engineering and is only worth it for very large archives.

For help on the wider structure of a news archive, see our news archive structure check.

Common mistakes

Four mistakes repeat across audits. The first is a canonical that points every slice to the topic root. The second is noindex on all pages after the first, usually added out of a fear of duplicates; Google's page mentions noindex only for unwanted variations of the same list. The third is a hidden pagination strip that exists on desktop only. The fourth is measuring scroll depth but not slice views, which leaves you blind to how the archive is used.

Rolling the change out safely

Converting a live archive is riskier than building a new one, because thousands of existing URLs are already in people's bookmarks and in search results. The following sequence keeps the risk contained.

1. Inventory the templates. Topic, author, tag and date archives often use different front ends. Note which scroll, which paginate and which do neither.
2. Add slice URLs first. Before touching the scroll script, make the server return a complete HTML page for each page number. This step is invisible to readers.
3. Add the links. Place the next-page link in the HTML beneath the list. Hide it visually only if the design demands it, and keep it in the document.
4. Fix canonicals and titles. Generate both from the page number.
5. Connect the History API. Update the address when a slice becomes the main visible content, then confirm that a refresh lands on the right slice.
6. Run the checklist on staging. Then inspect a few deep URLs on production with URL Inspection.
7. Watch coverage. Follow indexing and crawl reports and your server logs for several weeks. If something drops, check the canonical and the link output first.

We give no timeline, because Google's documents promise no recrawl schedule. What you can control is that the pages are ready whenever the crawl arrives.

Start with one section

Roll the change out to a single mid-sized section first, for example local news or one regular column, rather than the whole archive. If the section shows crawl or indexing problems, reverting is cheap. Keep the old URLs unchanged where possible. If a new URL scheme replaces the old one, prepare a 301 redirect map before launch, because a scheme change without redirects turns every old link into a dead end.

Where this fits in a technical audit

Treat this as one line item inside a broader technical review, not a standalone project. In our technical SEO audit checklist, crawl paths and canonicals come before layout details. A broken canonical template or a robots rule that blocks the feed API can matter more than the scroll behaviour itself.

A short freshness note: the recommendations in Google's documentation can change, so check the current pages before you implement. The status of rel="next" and rel="prev" is one detail that has already changed once.

Related guides: News Site SEO Checklist, Article Markup for News Sites: Date and Author Fields, AI and Scaled Content on News Sites: Where Spam Policies Draw the Line.

Conclusion

Infinite scroll does not hurt a news archive by itself. The damage comes when the scroll is the only way to reach older stories. Give each slice a persistent, unique URL, join the slices with real links, update the address with the History API, keep each page's canonical its own, and test with scripts off and with URL Inspection. For deep archives, numbered or date-based pages remain the lowest-risk choice. None of this promises a ranking; it only keeps your stories reachable.

If you are planning revenue around archive pages, you can look at the ad spaces on Akua Adforce; scope and terms are described on the product page.

For the primary sources, start with Google's pagination and incremental page loading (Opens in a new tab) page, the JavaScript lazy-loading (Opens in a new tab) documentation, the crawlable links (Opens in a new tab) guidance and Search Essentials (Opens in a new tab).

Frequently Asked Questions

Does infinite scroll stop Google from indexing older stories?

Not by itself, but it can if scrolling or clicking is the only way to reach them. Google's documentation says its crawlers do not click buttons and generally do not run JavaScript that needs a user action. Persistent URLs and real links remove the risk.

Is numbered pagination safer for a deep archive?

Generally yes. Each page has its own URL by nature, and Google's pagination page says that load more and infinite scroll cannot handle very large result sets. If you prefer a scroll experience, keep numbered links beneath it.

Can I use a hash in the URL for the page number?

No. The pagination documentation says Google ignores fragment identifiers and may skip such links. Use a query parameter like page=2 or a separate path.

Should page one be the canonical for the whole sequence?

Google's documentation advises against it. Give every page in the sequence its own canonical URL.

Does Google still use rel next and rel prev?

According to the pagination documentation, no. Other search engines may, but working links between pages are what matter for Google.

How do I test an infinite scroll archive?

Load a deep slice URL directly with scripts disabled, then check the rendered HTML in the URL Inspection tool for story links and the next-page link. The lazy-loading documentation recommends that tool for confirming content appears in the rendered HTML.