Traffic collapses: narrow the cause instead of guessing

On Monday the clicks are 30 percent below the previous week. Nobody published anything, nobody switched anything off, the home page looks the way it always does. What gets expensive is not the drop but the week in which everyone has a different theory and nobody has evidence — while every page that has carried noindex since the last deploy quietly keeps falling out of the index.

How to go about it in JMX

1. The History tab. Open the project file and hold the last run against the one before. The section What changed on the pages shows page by page what is new, gone or altered, with a risk per row: a page that has carried noindex since the previous run or answers with 404 is Critical and sits at the top. The tally is given per segment — shop or blog.
2. Search Console, then Search insights. Load data fetches clicks, impressions, CTR and position. Gained / Lost compares with the immediately preceding period of equal length; Site sections shows which URL area is losing.
3. The Technical tab. Eight core questions on one page: crawler access, robots.txt, sitemaps, indexability, redirects, load times, language codes and security headers. What could not be checked is called “Not checkable” there and not “Fine”.
4. Evaluate log files. Hold an Apache, nginx or IIS log against the crawl. Status (bot) next to Status (crawl) is the reason to read a log at all: where the column differs, the search engine is getting something other than you are. Only what reverse and forward DNS confirm counts as a bot.
5. Render JavaScript. Comparing the served with the rendered HTML finds the pages that set their noindex only in the browser — to every non-rendering crawler they look indexable.
6. Search Console, Inspect URLs. Query the real index status for the suspect pages, the only authoritative answer to “is this page in the index?”. The quota is 2,000 requests per day and property; JMX counts along and stops beforehand.
7. The Tasks tab. The work plan out of the findings: impact in points, estimated effort, leverage in points per person-day, a classification from Do now to Defer.

What to watch out for

In History, “gone” does not mean “deleted” but “not found in this crawl”. Change the maximum depth, the URL limit or the exclusion patterns between two runs and you get a list of vanished pages that says nothing about the site and everything about the changed setting. Without a license every analysis stops at 25 pages; comparing two truncated runs is no comparison.

JMX compares only what lies in the project file: if there is a single run in it, the first crawl after the drop is an inventory and not a cause, because there is nothing to hold it against. Search Console reaches back 16 months, the rank tracking history begins with the first paid run. A core update, a competitor overtaking you, demand that fell away: JMX says what changed on your site, not what changed in search.

The special case: the drop is not organic at all. Web analytics separates the channels, visits from AI apps without a referrer count as “Direct” by design, and the current month appears in no metric — half a month against a whole one would be an invented drop. Sometimes the counter is simply missing. The text search in does not contain mode, restricted to /shop, finds the pages without a tracking code.

Anyone looking for the cause of a drop is really looking for a second point in time. A nightly jmx-cli run against the same project file with --fail-on-change Critical supplies it and exits with return code 2 as soon as something critical has moved.