Daniel's personal website

search/About

This page is about search and searching on this website.
A seedful and seamful search interface

My goal here is to provide interfaces to my website that explore ["search seeds"](/search/?q="search+seeds") [@griffin2022situating] and _seamfulness_ [@eslami2016first]. I will try to expose points of interactions, talking explicitly about keywords and queries, suggesting search operators and keyboard shortcuts, and noting how search tools shape questions and content. I will try to see even results-of-search [@mulligan2018rescripting] as seeds for future querying. I will consider how to engage with [null results](\search\?q=sklf) and endeavor to support sharing and repairing searches and search. I will situate the searching and search tools.

How to search on this website?

See search/Guide.

How does search work on this website?

Search on this website is partially an experiment/exercise/exploration in using, filling, and explaining gaps in an outdated JavaScript client-side search library: Lunr.js (GitHub). This search function currently only largely provides 'site search', it only searches the pages of this website (except for that content not indexed, indicated below). It does not currently search the content of PDFs hosted on this website.

Added

Note: There is some minimal support for external searches, see ["External results"](#external-results) (below)

Added

Note: There is also a minimal working example of [FlexSearch](https://github.com/nextapps-de/flexsearch) running: [FlexSearch Working Example](https://danielsgriffin.com/ppost/2023/09/12/flexsearch-working-example.html)

Lunr.js

Lunrish

See: /changes/2023/12/11/notes/

noindex

stopwords

Words not indexed by Lunr.js. These words are not distinguishing when in a query unless it is an exact phrase search (see below).

Exact phrase searching

Lunr.js does not directly support exact phrase searching where spaces appear between search terms. Such searching is minimally made possible in an additional function (exactSearch) in js/search/results.js that conducts a simple includes check across the documents in item_index.json.

Sorting

Lunr.js does not directly support sorting. It is added in simple bespoke code in js/search/results.js and js/search/serp.js. See more at sort: in search/Guide/

Autocomplete

This is written in vanilla JavaScript in several scripts, including: 'autocomplete.js and 'suggestions.js. These scripts manage the autocomplete suggestions-list.

hand-curated-queries

Hand-curated queries () are drawn from Jekyll site.data.hand_curated_queries (and include also hand_curated_query_snippets). This is accessed in the JavaScript through Liquid syntax. These queries appear as soon as the search bar is in focus.

dynamic results

Dynamic results () are drawn from searching the Lunr.js index while the query is being formulated. These results appear as soon as one character has been typed in the search bar.

SERP (search engine results page)

"Exactly # results"

Notice this says "Exactly". This is a commentary on the confusion introduced by how Google reports the number of search results. The count of results by Google in its SERP—as interpreted by many searchers—is inaccurate. See [Randall Munroe for an explication on the XKCD blog: Trochee Chart](https://blog.xkcd.com/2011/02/04/trochee-chart/). See also [a longer explanation from Danny Sullivan from 2010](https://searchengineland.com/why-google-cant-count-results-properly-53559)—including Google's suggested justification for no disclaimer—where he shares Matt Cutts saying "We’ve talked about the fact that results estimates are just estimates for years" (Note: there is no contextual help to the searcher on the Google SERP that this is the case, despite what [Google's People + AI Guidebook](https://pair.withgoogle.com/) might say about Explainability + Trust).

hand-curated snippets

This label indicates snippets (for search results) that are manually written by the author. (These snippets are written directly into the page YAML.)

generated snippets

These snippets may be inaccurate. Generated snippets are marked with , include a tooltip explainer, and link here.

These snippets appear in the search results and in the hamburger list dropdown menu ( ) on the top left of each page.

I am implementing these to (loosely) explore the processes to develop and maintain them. This label indicates snippets in the search results that are generated by feeding strings of items in the item_index.json to OpenAI's 'gpt-3.5-turbo' model with a simple prompt (full code TK, slight modification of prompt for documents that were longer than the context window and so chunked):

# Task

Write a search snippet that briefly summarizes the following document.

# Content {text}

Example of an inaccurate snippet, for search/Guide:

This document is the official searching guide from Lunr.js and provides syntax for OR, AND, and NOT searches, as well as searching across specific fields. It also includes information on exact phrase searching, !bangs, wildcards, and boosts. generated snippet

External results

I'm experimenting with serving some external results, see [ type:external ]. If you own the result and do not wish to be included in my index, please tell me.

Similarity

Lunr.js does not directly support similarity searching. Similarity is determined here by running cosine_similarity (from sklearn) on embeddings of the pages across the website (in scripts/update_similars.py). The SERP will display the 30 "most similar" pages to the page queried. See more at similar: in search/Guide/.

"Similar" pages are also listed in the hamburger list dropdown menu ( ) on the top left of each page (via js/similars.js). Currently the five "most similar" pages are listed.