Adding Search to Hugo with lunr.js. Easy, fast, and configurable
I need search for my website
After a few months of adding content it became apparent I needed a search method. A few methods were available to me. I will categorize those methods as “local” and “3rd party”. Since Hugo is a static site generator, no methods for collecting user input or modifying the DOM were available. A few templates utilize third party search methods using google or other search sites. I wanted something local, configurable, and fast. Search is complex, choices range from building a index system using external systems like elastic to as simple as indexing tokens in the DOM. I choose to use the DOM and the JavaScript project lunr
What we need to get lunr working
Lunr was easy but required a series of steps to get working.
1. Lunr needs to build and index from your content, the content it indexes should be updated automatically.
2. Lunr needs input from a user and what they are searching for so it can look for results in your index.
3. Lunr needs a place to write the results to the searcher can access the content.
Building a index of site content
I choose to create a json representation of all site content. Hugo makes this easy with templates. Using the documentation for template lookup order from the hugo website I created a index.json at the root of my page. A few points about the below code. It is somewhat minified allowing for faster load time. The internal functions from hugo are used to quote and collapse content (jsonify, plainify). I needed specific sets of tokens indexed, making sure that all content, tags, and defined keywords were included in the json.
index.json
Filename:index.json{ "items" : [
{{- $pages := $.Site.RegularPages -}}
{{- $cnt := 0 -}}
{{- range $pages -}}
{{- $cnt = add $cnt 1 -}}
{"url" : "{{ .Permalink }}","title" : "{{ .Title }}","date" : "{{ .PublishDate.Format "January 2006" }}","content": {{ .Content | plainify | jsonify }},"tags": {{ .Params.tags|jsonify }},"keywords": {{ .Params.keywords|jsonify }} {{ if eq $cnt (len $pages) }}}{{ else }} },{{- end -}}{{- end -}}]}UPDATE The above json, although works and was in production for multiple years. A recent revisit to the code while optimizing template build times forced a rewrite of the index.json file.
The changes I applied are:
- Remove the minification in the json template itself. Allowing the template to be easily read. The build option –minify when building (see output versions) the site automatically minifies the index.json that is created by this template.
- In the loop we keep track of how many pages we must create an object for and delimit the json with a comma. The code above will ask for the length of the pages with each iteration. We can move the length call outside the loop and get the same functionality without the overhead of calling length on Pages each iteration.
index.json{"items" : [
{{- $pages := $.Site.RegularPages -}}
{{- $totalPages := len $pages -}}
{{- $cnt := 0 -}}
{{- range $pages -}}
{{- $cnt = add $cnt 1 -}}
{"url" : "{{ .Permalink }}",
"author" : "{{ .Params.author }}",
"title" : "{{ .Title }}",
"date" : "{{ .PublishDate.Format "January 2006" }}",
"content": {{ .Content | plainify | jsonify }},
"tags": {{ .Params.tags|jsonify }},
"keywords": {{ .Params.keywords|jsonify }}
{{- if eq $cnt $totalPages -}}
}
{{- else -}}
},
{{- end -}}
{{- end -}}]}Build a section for user input
Building a section related to search was easy. Make a directory named “search” in your content root and create a template based upon lookup order to load for the section.
I put the following in layouts/_default/search.html then I just use ’layout: search’ in the frontmatter to load the page content
Filename:search.html{{ define "main" }}
{{ if .Params.lunr }}
{{- partial "lunr.html" -}}
{{- end -}}
<div class="col-lg-8 offset-lg-12 px-lg-14">
<section class="post">
<h1 class="post-title">Search all published documents:</h1>
<input type="text" id="sinput" placeholder="" oninput="showSearchResults()">
<br />
<ul id="list"></ul>
</section>
</div>
{{ end }}Section Search (IE: Making a directory/section to load the search page)
Filename:content/search/\_index.md---
layout: "search"
title: "Search"
description: "Search all content"
lunr: True
date: "2024-08-06T00:00:00+00:00"
---load lunr.js and write content
I downloaded lunr.js to my local assests directory in my theme. I decided to load lunr.js via a front matter variable “lunr: True”. This variable allows me to define on any page if I want to have lunr.js loaded in a minfied way.
I currently have the following functions built into lunr-udf.js. We can call it via resources.Get and minify it, then bundle it with the lunr.js. All using the lunr.html file in the partials directory
Filename: lunr-udf.js
var searchElem = ((document.getElementById("sinput")||{}).value)||"";
var posts;
function loadSearch() {
// call the index.json file from server by http get request
var baseName = window.location.origin;
var path = "/index.json";
var xhr = new XMLHttpRequest();
xhr.onreadystatechange = function () {
if (xhr.readyState === 4) {
if (xhr.status === 200) {
var data = JSON.parse(xhr.responseText);
if (data) {
posts = data.items; // load json data
}
} else {
console.log(xhr.responseText);
}
}
};
xhr.open('GET', baseName+path);
xhr.send();
}
if (location.pathname == "/search/") {
if (document.readyState === "loading") {
document.addEventListener("DOMContentLoaded", loadSearch())
} else {
loadSearch();
}
}; // call loadsearch to load the json file if in the search path
function showSearchResults() {
var query = searchElem = ((document.getElementById("sinput")||{}).value)||"";
var searchString = query.replace(/[^\w\s]/gi, ''); // clear white spaces
var target = document.getElementById('list'); // target the ul list to render the results
var postsByTitle = posts.reduce((acc, curr) => { // map lunr search index to your articles
acc[curr.title] = curr;
return acc;
}, {}
);
// build lunr index file
var index = lunr(function () {
this.ref('title')
this.field('content')
this.field('tags')
this.field('keywords')
posts.forEach(function (doc) {
this.add(doc)
}, this)
});
// search in lunr index
if (searchString && searchString != '') {
var matches = index.search(searchString);
var matchPosts = [];
matches.forEach((m) => {
matchPosts.push(postsByTitle[m.ref]);
});
if (matchPosts.length > 0) {
// match found with input text and lunr index
target.innerHTML = matchPosts.map(function (p) {
if (p != undefined) {
var baseName = window.location.origin
if (p.tags != undefined) {
var txtTag = p.tags.map(function (t) {
return `
<a href="${baseName}/tags/${t}"
class="tag bg-gradient text-dark">${t}</a>`
}).join(' ').toLowerCase()
}
return `<div class="summary><li>
<span class="date">${p.date}</span> -
<a class="tag bg-gradient text-dark" href="${p.url}"> ${p.title}</a>
<b>  Tags:</b>${txtTag}</div>`;
}
}).join('');
} else {
// if no results found, then render a general message
target.innerHTML = `<br><h2 style="text-align:center">No search results found</h2>`;
};
} else {
target.innerHTML = `<br><h2 style="text-align:center">No Input</h2>`;
}
};I did not want to deliver a index.json xhr call on each page, so I check to see if the user is on the path /search/ before sending a xmlHTTPRequest. Directing the input to the target allows for the search list to be dynamically loaded by the browsers dom when search queries have been executed.
Adding lunr.js to any page with front matter variable
Create a lunr.html partial. I use the hugo native minify to make lunr.js small. ~66% savings
Make the file lunr.html in the partials directory (Lunr partial)
Filename:partials/lunr.html{{- $lunrSrc := resources.Get "lunr.js" | minify -}}
{{- $udfLunr := resources.Get "lunr-udf.js" | minify -}}
{{- $lunrBundle := slice $lunrSrc $udfLunr | resources.Concat "lunr.js" -}}
<script defer src="{{- $lunrBundle.RelPermalink -}}"></script>If you would like email or rss feed updates. Visit the Subscribe page
Share with others