How sites are listed, held and removed

What is listed

The Tonvin Collection lists websites that the people behind it build and run. It is not a submissions directory: there is no form or process for proposing a site.

Every site is a row in one table, and each row says whether it is listed in the directory or kept private. The directory pages, search, the sitemap, the feeds and the machine-readable files read only the rows marked as listed. The code that reads the table is written so that forgetting to ask for listed rows shows too little rather than too much, and a check fails the build if an accessor skips the filter.

• Private and client sites are counted in the totals the collection reports. On these pages they are not named or linked.
• Sites that run entirely at the edge, as Cloudflare Workers or Cloudflare Pages, are listed on the Edge sites page and nowhere else: not in the directory, search, the sitemap, llms.txt or the feeds. The few of them with adult content are labelled as such there.
• The collection also runs a group of WordPress sites and community forums. They are counted in the totals, not listed one by one, and the fleet page for them reports counts only.

How a site is added

The website itself never writes a listing. The person who runs the collection adds a row by running a SQL script against the database, so a site appears in the directory only when the operator puts it there.

The tagline and description are drafted from what the live site says about itself, including its own llms.txt where it has one, and screenshots are captured from the live site. The draft is checked before it goes live. Each directory site is filed under one of the specialist categories.

When a site is held, skipped or limited

A site is not added to the directory while it is down or returning errors, or while it carries a takedown notice. A site that is still closed to search engines ahead of launch can be listed, marked as pre-launch until it opens. Sites with adult content are not added to the directory.

• A case study for a site that is still before launch is held back. It has no page, no sitemap entry and no link until the site opens to search engines.
• A site that is closed to search engines before launch is not given a CiteFiles grade. It is marked as pre-launch.
• A language gets its own page in the sitemap only when at least three of the listed sites publish in it.
• The showcase deck opens with a hand-picked selection of sites. Everything else follows, largest first.

What is measured once a site is listed

Every night a check opens each site and judges it by the text it renders, not only by its status code. A change in a site’s state is reported to the operator as a change, not as a nightly list of the same faults.

Every night the feeds that the sites publish are read, and each site’s sitemap is looked at to find when it last changed. A sitemap date is trusted only when it looks like real per-page editing. A bulk build stamp applied to every page is discarded.

Screenshots are re-captured on a nightly rotation, oldest first, and a picture is replaced only when it has visibly changed.

Once a week each site’s page count is re-measured from the site’s own sitemap and kept as a history. A very large site is not re-walked and keeps the last full count it was given. A sitemap that cannot be read is not recorded as zero.

The nightly jobs do not rewrite a site’s tagline or description. Those change only when a person reviews them. A figure that has not been measured is left out rather than shown as zero.

The measured figures are on the Measured page, the infrastructure figures are on the Stack page, and what changed is on the Changes page.

What the CiteFiles ratings are

Many listings carry an AI-SEO grade from A to F, worked out from a score between zero and one hundred. The grade comes from CiteFiles, a scanner in this collection. CiteFiles is itself one of the collection’s sites, so the rating is not an independent third party’s, and every place that shows a grade says so.

Each grade carries the date it was scanned and the version of the scanner that produced it. A site that has not been scanned shows no grade. The grades are kept as a stored snapshot that is refreshed by re-running the scan and importing the result, so they are not live. The scoring is described in the CiteFiles methodology.

Reusing the catalogue

The catalogue data carries one licence statement, the same line the machine-readable index sends in its licence field: “Tonvin Collection catalogue data — free to reuse with attribution to https://tonvincollection.com”.

That index is at /api/sites.json. It lists the sites in the directory with their category, languages and page counts.

Corrections and removal

Anyone may ask for a correction to a listing, or for a site to be removed from the collection. Use the contact page and say which site or page you mean and what is wrong. Requests are reviewed within 7 days.

The directory pages, search, the sitemap, the feeds and the machine-readable files are all built from the same listing, so a correction or a removal is made once, in the listing.