This is an example using ElasticSearch with JQuery and Python, I built that concept using synonyms in Windows. Thanks to my friend Ricardo for your support ;)
The challenge was to build a solution that would be able to search by a word to return the synonyms like that:
To build it, I needed to set up the environment, so I followed it up:
1º - INSTALL THE ELASTICSEARCH
I Installed the Windows version for a pure concept.
2º - INSTALL SENSE
It's a helpful tool for testing and development
3º - INSTALL ELASTICSEARCH HEAD
A relevant tool to check the indexes, build queries.
4º - INSTALL ELASTICSEARCH TOOLBOX
Another relevant tool to check the indexes and validate the connections.
5º - ELASTIC SHARED ACCESS PERMISSION
To all the users be able to access the Elastic, it's necessary to change the Elastic configuration file located on C:\ProgramData\Elastic\Elasticsearch\config\elasticsearch.yml
6º - REMOTE ELASTIC CONFIGURATION
To make possible the access out of the box (localhost), it's necessary to change the host configuration:
7º - CONFIGURE THE SYNONYMS
To make the search much more powerful, I configured the synonym files (next step), I used the website called: www.thesaurus.com for it:
8º - CONFIGURE THE SYNONYM FILES
I created a folder named analysis, it must be inside the ES configuration path, in my case it was: C:\ProgramData\Elastic\Elasticsearch\config\analysis
9º - CONFIGURE THE WINDOWS IIS
I used IIS because I'm using Windows, it made a difference when I needed to test JQuery:
10º - CONFIGURE ELASTICSEARCH FOR SYNONYMS
First of all, we need to set up ES to use the synonyms using Sense:
PUT /concept_index
{
"settings": {
"index" : {
"analysis" : {
"analyzer" : {
"car_standard_synonyms" : {
"type": "custom",
"tokenizer" : "standard",
"filter" : ["lowercase", "synonym_filter"]
}
},
"filter" : {
"synonym_filter" : {
"type" : "synonym",
"lenient": true,
"synonyms_path":"analysis/synonyms.txt"
}
}
}
}
},
"mappings": {
"car": {
"properties": {
"model": { "type": "text", "analyzer": "car_standard_synonyms" }
}
}
}
}
11º - LOAD THE DATA FROM MSSQL SQL TO ELASTICSEARCH
I used Python to do it, it took 2 seconds to load 3500 records, this is the code:
import pandas as pd
import pyodbc
import elasticsearch
es = elasticsearch.Elasticsearch("https://localhost:9200/")
cnn = pyodbc.connect('DRIVER={ODBC Driver 13 for SQL Server};SERVER=localhost;DATABASE=MyConceptDB;Trusted_Connection=yes')
strSQL = "select ID, Model from [dbo].[CarIndex]"
cursor = cnn.cursor()
cursor.execute(strSQL)
#df = pd.read_sql(strSQL,cnn)
tables = cursor.fetchall()
for row in tables:
index_name = 'standard_web_index'
doc_type = 'car'
doc = row.Model
doc_id = row.ID
es.index(index=index_name, doc_type=doc_type, body={'model': doc}, id=doc_id)
import pandas as pd
import pyodbc
import elasticsearch
es = elasticsearch.Elasticsearch("https://localhost:9200/")
cnn = pyodbc.connect('DRIVER={ODBC Driver 13 for SQL Server};SERVER=localhost;DATABASE=MyConceptDB;Trusted_Connection=yes')
strSQL = "select ID, Model from [dbo].[CarIndex]"
cursor = cnn.cursor()
cursor.execute(strSQL)
#df = pd.read_sql(strSQL,cnn)
tables = cursor.fetchall()
for row in tables:
index_name = 'standard_web_index'
doc_type = 'car'
doc = row.Model
doc_id = row.ID
es.index(index=index_name, doc_type=doc_type, body={'model': doc}, id=doc_id)
12º - FINALLY - JQUERY TO SEARCH ELASTICSEARCH
I saved the JQuery inside of the index.html file C:\inetpub\wwwroot:
<html>
<head>
<link rel="stylesheet" type="text/css" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
<link rel="stylesheet" href="https://code.jquery.com/ui/1.9.0/themes/smoothness/jquery-ui.css" />
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/jqueryui/1.12.1/jquery-ui.css" type="css">
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.1.1/jquery.js"></script>
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/jqueryui/1.12.1/jquery-ui.js"></script>
</head>
<body>
<script>
$(function() {
$("#keyword").autocomplete({
source: function(request, response) {
var postData = "q=model:"+ request.term.toLowerCase()
$.ajax({
url: "https://tg-pre-sql4:9200/standard_web_index/_search",
method: "GET",
crossDomain: true,
dataType: "JSON",
data: postData,
success: function(data) {
response($.map(data.hits.hits, function(item) {
return {
label: item._source.model,
id: item._source.id
}
}));
},
});
},
minLength: 1
})
});
</script>
<h2><center>
<img src="logo/esitconcept.jpg">
<p>
<br>
<br>
<div class="ui-widget" align="center" border=1>
<label for="keyword"><font size=5>Keyword:</font> </label>
<input id="keyword" size=50>
</div>
</center></h2>
</body>
</html>
