Caching SearXNG

This revision is from 2024/03/17 11:06. You can Restore it.

SearXNG installs itself on /usr/local/searxng/searxng-src, with the main source code in searxng-src directory.

To hack the results, the file is webapp.py in /usr/local/searxng/searxng-src/searx/webapp.py

The function in webapp.py is...

@app.route('/search', methods=[['GET', 'POST']])

def search():

One sqlite3 database. Named: searxng_distributed_cache.db3

  1. stores the search terms
  2. stores the urls

The database is distributed, the keywords have to go to maintainers, they perform searches, robot queries build a more comprehensive database.

CREATE TABLE IF NOT EXISTS search_results (

id INTEGER PRIMARY KEY AUTOINCREMENT,

search_term_id INTEGER,

url TEXT,

title TEXT,

content TEXT,

img_src TEXT,

engine TEXT,

parsed_url TEXT[],

template TEXT,

engines TEXT[],

positions INTEGER[],

published_date DATE,

score REAL,

category TEXT,

pretty_url TEXT,

pubdate DATETIME,

open_group BOOLEAN,

FOREIGN KEY(search_term_id) REFERENCES search_terms(query) ON DELETE CASCADE

);

and

CREATE TABLE IF NOT EXISTS "search_terms" (

"id" INTEGER PRIMARY KEY AUTOINCREMENT,

"query" TEXT UNIQUE

);

Proposed searXNG options:

  • use cache
  • update the cache
  • disclosure to end user

Benefits:

  • turns searXNG into a full search engine built from caching results
  • searches are against a local file, so it speeds up searching significantly
  • offline searching

Refactoring def search():

Step 1: Does the search term exist in the database?

  

📝 📜 ⏱️ ⬆️