Hulipaa is a great library that allows you to have a search engine in your website without needing a backend server running.
In this page we'll see an example on how to set up Hulipaa.
To look at the final code you can see it in this github repo: Hulipaa example
In this tutorial we assume that you have already installed npm and node locally and created an npm project.
Usually when you start using Hulipaa you'll already have a static site and you want to add a search functionality to it.
If that's the case you can jump directly to the next section.
But if you don't have one and you want to follow the tutorial anyway we'll create a very basic static site generator in this section.
If you don't already have an npm project initialised or it doesn't contain a package.json file, you can create one with the command
npm initWhen the terminal asks for it, give a name and a version to your project. Usually it's fine to just use the default values for all the prompts.
We'll need a new folder in the root of your project called data that will contain the data to show on your web pages.
We need to create another folder called pages that will contain the html pages to show to the user.
These pages will be automatically generated by our script using the data in the data folder.
Now we can create a new file called index.js that will use the data inside our data folder to automatically create web pages.
Inside the script we can add this code:
const fs = require('fs')
const path = require('path')
const dataFolder = 'data'
const pagesFolder = 'pages'
// Clean pages directory
if (fs.existsSync(pagesFolder)) {
fs.rmSync(pagesFolder,{ recursive: true })
}
fs.mkdirSync(pagesFolder)
// Generate web pages
const dataFiles = fs.readdirSync(dataFolder)
const createdPages = []
for (let dataFileName of dataFiles) {
const dataFullPath = path.join(dataFolder,dataFileName)
const dataFileContent = fs.readFileSync(dataFullPath,'utf8')
const dataOfPage = JSON.parse(dataFileContent)
const { title,content } = dataOfPage
const html = `<html><head><title>${title}</title></head><body><h1>${title}</h1>${content}</body></html>`
const pageName = `${path.parse(dataFileName).name}.html`
const pageFullPath = path.join(pagesFolder,pageName)
fs.writeFileSync(pageFullPath,html)
createdPages.push([pageName,pageFullPath])
}
// Generate main web page
let htmlFiles = fs.readdirSync('.').filter(fileName => path.extname(fileName) === '.html')
htmlFiles = htmlFiles.filter(fileName => fileName !== 'index.html')
const htmlListHtmlFiles = htmlFiles.map(fileName => `<li><a href="${fileName}">${fileName}</a></li>`)
const htmlListGeneratedPages = createdPages.map(([page,path]) => `<li><a href="${path}">${page}</a></li>`).join('')
const htmlIndexFile = `<html><head><title>Main page</title></head><body>
<p>Html Pages</p><ul>${htmlListHtmlFiles}</ul>
<p>Generated Pages</p><ul>${htmlListGeneratedPages}</ul>
</body></html>`
fs.writeFileSync('index.html',htmlIndexFile)After this everything should be in place to have our static site generator. We just need to create a few files in the data folder to have some pages we can use.
Create a first file called page1.json with this text inside:
{"title": "Page 1", "content": "The content of the first page"}And then create a second file called page2.json with this inside:
{"title": "Page 2", "content": "The content of the second page"}And then run the index.js script by typing in your terminal
node index.jsNow your site is fully created, you just need to serve the website with a simple http server and then open the index.html file.
If you don't know how don't worry, we'll do it later in the tutorial.
Hulipaa is made out of 2 parts, the first is used during the generation of the website and the second one is used in the website itself when it's opened in the browser.
In this section we'll set up the first part, that will generate all the results the user might search for.
First we need to install the library in our npm project.
All we need to do is to type in a terminal
npm install --save-dev hulipaaNext we need to import the library.
Hulipaa supports both ES Modules and CommonJS modules. In this tutorial we are going to use the CommonJs syntax, but the same result can be achieved using ESM.
The library exports just one big top level function. So you can import it like this
const Hulipaa = require('hulipaa')If you have created the project from scratch using this tutorial, remember to add this code at the top of the index.js file.
Now we can call the library to generate the search results.
Hulipaa needs 4 properties to be defined when it's called, they are needed to find, load each page, and generate the results used by the searchbar.
So calling the library will look something similar to this
Hulipaa({ inputFolder, parseData, generateLink, outputFolder})Below there is an explanation of each property.
The inputFolder property is a path to a directory. This directory should contain the data that you use to generate each page in your website.
Hulipaa will open this directory and load any file inside it, the content of the file will then be sent to the parseData function to be parsed.
The parseData property needs to be a function and is used by Hulipaa to get the details of each page.
This allows Hulipaa to be very flexible and support any type of data format.
Hulipaa will call this function once for each file present in the inputFolder directory. The function will receive 2 parameters: the content and the path of the file
function parseData(fileContent, filePath){}The function should parse the content of the file and then return an object containing 3 properties:
- text: The text that should be visible and searchable by the user. This should be plain text and shouldn't contain any markup language.
- title: The title of the page, this will be shown to the user when they see the results of a query.
- path: The path to this file, reachable by the browser. This means that each file inside the
inputFoldershould be made available by the web server. Keep in mind that the search page should have access to this path. So depending on how the website is set up and where your search page is, you might need to adjust the path.
function parseData(fileContent, filePath){
// ... [Parsing data]
return {text, title, path}
}The generateLink property is another function. It will be used by the library to create the link of each result shown in the search page.
Hulipaa will call this function passing the name of each file inside the inputFolder directory
function generateLink(fileName, inputFolder){}The link that this function returns will be used to open a page when the user clicks on a search result. This means that it should point to the page created by the website generator.
function generateLink(fileName, inputFolder){
// ... [Creating link]
return link
}I suggest each link is an absolute path instead of a relative one, this is to avoid issues related to the position of the search page.
The outputFolder property is the path to a directory that will be used to store the search results that are generated.
The directory is refreshed each time we re-generate the results. Therefore it shouldn't be used for anything else.
Now that we know all the parameters we can use them to execute the library and generate the search results.
If we take as example our project we created in the first section, we would call the library using these parameters:
Hulipaa({
inputFolder: 'data',
parseData: function (fileContent, filePath){
const parsedFile = JSON.parse(fileContent)
return {
text: parsedFile.content,
title: parsedFile.title,
path: filePath
}
},
generateLink: function (fileName, inputFolder){
const nameWithoutExtension = path.parse(fileName).name
return `/pages/${nameWithoutExtension}.html`
},
outputFolder: 'search'
})This needs to be executed after the website is generated, in our example that would be at the bottom of our index.js file.
Now all we need to do is to run the script that calls the library. Following our example above we just need to type in the terminal
node index.jsThis will scan through all the files and generate the search results, indexing all the word inside each file present in the inputFolder.
Now the results are ready to be used so we'll need the second part of the library.
First we need somewhere to show the searchbar so the user can search through your website.
You might already have a page where you want to put the searchbar, but if you don't have one yet we can create a simple one now.
Following our example website, we'll create a file called search.html in the root directory of our project, and we'll fill it with a barebone html structure:
<html>
<head>
</head>
<body>
</body>
</html>To import the library all we need to do is to add this line in the head section of our html
<script type="module" src="https://cdn.jsdelivr.net/npm/hulipaa-ui@1.0.2"></script>When Hulipaa is activated it will need a place to put the searchbar and later on the search results.
So we'll need to add the following in the body section of our page
<div class="hulipaa_searchbar"></div>The library will automatically recognise the hulipaa_searchbar class and use the div. This means that the searchbar can be used in a page that already contains other content.
You can also style the div to control how much space it will occupy. By default it will always fill whatever space is available in the parent element.
The library will not create or modify any html that is outside the hulipaa_searchbar element.
The last thing to do is to initialise the library.
But before doing that we'll need to know some properties that the library will use.
The first property is resultsPath which is the path to the folder where we generated our search results.
This means that it's the same directory that we specified in the outputFolder parameter when we used Hulipaa while building our website.
But remember that this path needs to be reachable from the browser, so depending on how the website is deployed we might need to tweak the path. For this reason I suggest you specify an absolute path, because a relative path might change depending on the location of the search page.
The parsePage property is very similar to the parseData property we used before. It parses a page and returns it's content.
Hulipaa will call this function each time it needs to visualise a search result. It will pass in the content of the data file.
The function should return an object with the text property, containing the text of the page in plain text and without markup language. This will be used to show a preview of each search result.
The parsePage property is intentionally compatible with the parseData property. Therefore you can reuse the exact same function and avoid any dupliacted code.
When we imported the library, a global function was created called Hulipaa. We can call that function when our page is fully loaded and it will create the searchbar for us.
So following our example, we can add this in the head section of our search.html page
<script type="text/javascript">
window.addEventListener('DOMContentLoaded',() => {
Hulipaa({
parsePage: function (fileContent){
const parsedFile = JSON.parse(fileContent)
return {
text: parsedFile.content
}
},
resultsPath: '/search'
})
});
</script>Remember to make sure the hulipaa_searchbar element has been loaded before calling the library. Usually you can achieve this by placing the script at the end of the body section, or by using the DOMContentLoaded event, like we did in the above code.
Now everything is ready, the website is finished and all we need to do is to open the main page of our website.
If you had created your site before this tutorial then you'll probably already have a webserver that will let you open it.
But if that's not the case we can easily install one, just type in a terminal
npm install --save-dev live-server
npx live-serveror, if the second command doesn't work, you can try
node ./node_modules/.bin/live-serverNow you can open the browser and load your website, if you used live-server the url will be http://localhost:8080.
Navigate to the search page, and the seachbar should be fully loaded.
The searchbar will look similar to the one here
You can search anything you want, remember that the example website we created contains words like content, page, first.
You should be able to see the title and a preview of each page, and by clicking on the title of the search result you will be redirected to the page.
Hulipaa can be a great search engine for your static website.
To set it up you just need to
- Install and import the library
- Specify where the webpages are and how to read them
- Generate the search results alongside your website
- Add the library to your search page
- Specify where the search results were saved
- Activate the library in the frontend
Remember that Hulipaa can handle any amount of data while keeping a light footprint on the browser.
And if you have any issues you can read the Troubleshooting page, or raise an Issue on github.