A Personal Search Engine (PSE) is a search engine that specializes in your interests. It provides personalized search results by indexing only the things you are interested in, not by spying on you. Instead of crawling the entire Web and then looking for what you've searched for, A PSE crawls only the parts of the Web you are most interested in and looks for what you're searching for there. The result is a list of hits that are relevant to your interests that point to websites you are more likely to know and trust.
A PSE can run locally (on the same computer you use to browse the web) and in fact I expect this is how most of them will be used. A local PSE is exceptionally fast and by it's nature exceptionally private.
There are reasons you might consider deploying a PSE to a public server, but I'll get into later.
A PSE starts with an empty index and will need to be "seeded" before it can return any results on it's own. The simplest form of seeding is feeding the PSE the domain names of a websites you trust. Other sources of trusted domains can be fed to the PSE, such as a list of bookmarks or a collection of RSS feeds.
Any source of trusted domains can be used to seed or add to the index. The important thing is to tailor this list to websites that focus on your interests, the types of things **you** want to see in the results. If you don't want spam or ads or computer-generated content in your results, don't add domains with that sort of thing to your PSE. Additionally, the more specialized the domain the better; a website like "vintageapplecomputers.com" is more likely to provide high-quality results to a search like "Macintosh SE/30" than "facebook.com", etc.
Once a PSE is seeded it can begin returning search results. The act of searching causes the PSE to "self-seed" by crawling for more domains to index within the pages you select from the results. These additional domains could be added to the index automatically, or they could be curated in a list of "discovered" domains for later manual review.
When what you're looking for can't be found in the local index, a PSE can "federate" with the PSE's of people you trust to broaden the search.
This could continue, with PSE's calling "friends of friends" to build a wider network but I'm not sure this is necessary and there are some undesirable side-effects to this sort of multi-hop federation.
When you reach the end of the local search results, results from the PSE's you federate will follow.
Federation can also be used to "bootstrap" a new PSE. If a list of trusted PSE's is provided, searching the federation will provide results even before the local index is seeded. Selecting websites from these results will begin the self-seeding process described earlier.
Using federation this way has benefits beyond increasing search results. It spreads the work of searching, storing, maintaining indexes out across many independent nodes, making the cost of operating a large search index lower (low enough for individuals to contribute meaningfully to a global index). It also makes the system fast without having to run on expensive hardware.
Any PSE can utilize the federated network, (i.e.: use it to get results) but in order to contribute, a PSE needs to be reachable from the nodes it federates with. In a perfect world, PSE's running locally would be able to talk to other PSE's running on other people's computers, but right now it's more likely that a fully-federated PSE will need to run on a publicly-addressable server. A well-designed PSE will be capable of "graduating" from running locally on someone's computer to a publicly-hosted node capable of contributing to the federation.
Publicly-hosted PSE's could lead to a handful of servers handling all requests, and that would undermine the purpose of PSE's. Ideally there is a 1:1 ratio between people and PSE's. This keeps the local results tightly bound to a given person's interests and keeps the computing resources required for an individual node light. I'm not sure how to avoid this other than the "natural consequences" of running such a large node which include less-relevant search results, slower searching and increased resource consumption. Hopefully pain like that is enough to keep this risk at bay.
I've built a prototype PSE based on the description above and used it for about a week to guage both it's usefulness and to understand how much effort is required to create it. I'm excited to say that the results are positive on both accounts.
The prototype was used to index a few hundred domains locally (seeded from my RSS feeds) and both the speed (<4ms average) and quality of the results were exceptional. Resource consumption of the prototype is quite reasonable, using less than 10MB of memory to search and maintain a 5GB index.
I also tested federated modes (LAN only, nothing deployed publicly yet) and it works surprisingly well with a very simple federation implementation. Federated result latency will of course be greater across the Internet, but they are capped at a (configurable) 3 seconds so that would be the worst-case regardless of network and remote node latency.
I had planned to setup a public node and share the prototype with some friends this week, but due to The Month of Zig I'm going to delay that in order to rewrite the prototype in Zig. I don't know how long this will take, but I wouldn't expect it this month, and probably not until after July. If you're interested in helping experiment with the prototype when it's ready, get in touch.
Jason J. Gullickson, 2026