Learning Elasticsearch with Laravel

In this post we’ll explore Elasticsearch; the basics of search and how to set it up with Laravel, Homestead and even Forge. Even though there are resources out there,  I couldn’t find the one article that summarised everything I needed to get up and running in as short a time as possible.

What is Elasticsearch and why should I use it?

Elasticsearch is a full text search server. You install it on your box and communicate with it via an HTTP Rest Api. Once up and running, you push data to it that you want to search through later.

It allows you to search through vast amounts of data in a very fast way with a feature rich query mechanism.

Even though you can perform fulltext searches on SQL databases, it isn’t as powerful or scalable as Elasticsearch. One benefit of using Elasticsearch is its schemaless approach that allows you to store data in a flat way, allowing you to transform complex joined entities into a single object. Remember that you can still create relations and nested objects if you need to.

Some key features that interest me are the abilities to perform “fuzzy” searches that allow for misspellings and a powerful relevance scoring mechanism, as well as the ability to retrieve the “next best” results when an exact query comes up with no matches.

Installing Elasticsearch

I’ve created a bash script to install the latest version of java and Elasticsearch here http://forgerecipes.com/recipes/73.

You can install Elasticsearch on homestead by SSHing into your homestead environment, and pasting that script into your terminal. When it’s done you should have java and Elasticsearch installed and running! You can also save the script as a recipe on Forge to allow you to easily deploy Elasticsearch to your servers.

Run “curl ‘localhost:9200” on homestead to ensure that Elasticsearch is working.

Talking to Elasticsearch with PHP

Now that our environment is set up, we can start using Elasticsearch! The official PHP client is on github and can be installed via composer:

composer require elasticsearch/elasticsearch

While you can use Elasticsearch’s Client and ClientBuilder classes directly, let’s wrap them in our own class. This way we can build an Api that is fluent for us and if the Elasticsearch package updates with breaking changes, we only have to fix our wrapper.

A basic wrapper might look like the below, and I’ve created a slightly beefed up version here.

<?php

namespace App\Elastic;

use Elasticsearch\Client;

class Elastic
{
    protected $client;

    public function __construct(Client $client)
    {
        $this->client = $client;
    }

    /**
     * Index a single item
     *
     * @param  array  $parameters [index, type, id, body]
     */
    public function index(array $parameters)
    {
        return $this->client->index($parameters);
    }

    /**
     * Delete a single item
     *
     * @param  array  $parameters
     */
    public function delete(array $parameters)
    {
        return $this->client->delete($parameters);
    }

    public function search(array $parameters)
    {
        return $this->client->search($parameters);
    }
}

To get this to work, we need to bind our dependencies in a Service Provider. I’ve also set up some basic logging to be able to more easily debug problems in the code snippet below.

public function register()
{
    $this->app->bind(Elastic::class, function ($app) {
        return new Elastic(
            ClientBuilder::create()
                ->setLogger(ClientBuilder::defaultLogger(storage_path('logs/elastic.log')))
                ->build()
        );
    });
}

At this stage we’re able to new up instances of our Elastic class through Laravel’s Service Container and interact with Elasticsearch!

$elastic = app(App\Elastic\Elastic::class);

$elastic->index([
    'index' => 'blog',
    'type' => 'post',
    'id' => 1,
    'body' => [
        'title' => 'Hello world!'
        'content' => 'My first indexed post!'
    ]
]);

Indexing

Elasticsearch organises data into indexes and types which can be likened to SQL’s databases and tables. Before we can utilize Elasticsearch’s powerful search features, we’ve got to populate it with some data.

We can hook into our Eloquent Model’s saved and deleted events to keep Elasticsearch in sync with our database.

<?php

public function boot()
{
    $elastic = $this->app->make(App\Elastic\Elastic::class);

    Post::saved(function ($post) use ($elastic) {
        $elastic->index([
            'index' => 'blog',
            'type' => 'post',
            'id' => $post->id,
            'body' => $post->toArray()
        ]);
    });

    Post::deleted(function ($post) use ($elastic) {
        $elastic->delete([
            'index' => 'blog',
            'type' => 'post',
            'id' => $post->id,
        ]);
    });
}

While a one time import might look like this:

$elastic = $this->app->make(App\Elastic\Elastic::class);

Post::chunk(100, function ($posts) use ($elastic) {
    foreach ($posts as $post) {
        $elastic->index([
            'index' => 'blog',
            'type' => 'post',
            'id' => $post->id,
            'body' => $post->toArray()
        ]);
    }
});

Searching 

Now that we’ve indexed some data, we can finally search through it! Let’s assume that we’ve got a single text field search box and we want to return some relevant data.

Elasticsearch has a vast query language. For this example I’d like to accomplish 2 things from our search:

  • Search multiple fields from our posts with relevance
  • Allow for misspellings in the search term and even our post content

The search parameters will look like this:

$parameters = [
    'index' => 'blog',
    'type' => 'post',
    'body' => [
        'query' => $query
    ]
];

$response = $elastic->search($parameters);

Let’s populate our $query variable. Out of the box, Elasticsearch returns results sorted by relevance. We can search through multiple fields using a multi_match search, which allows fields to be given a weight that gives them more importance than other fields.

$query = [
    'multi_match' => [
        'query' => $search,
        'fields' => ['title^3', 'content'],
    ],
];

Allowing for misspellings means utilizing Elasticsearch’s “fuzzy” search features. Without going into any detail, we can enable this by adding a single line:

$query = [
    'multi_match' => [
        'query' => $search,
        'fuzziness' => 'AUTO',
        'fields' => ['title^3', 'content'],
    ],
];

And that’s it! Our Elasticsearch instance will respond with our search results ordered by relevance, and allow for misspellings in the search query.

There’s a lot to learn when it comes to Elasticsearch, and the sample code provided needs to be improved for use in production – but with this knowledge you should be able to quickly get yourself into a position to explore Elasticsearch.

Resources

Here are a number of resources that I used when learning about Elasticsearch.

Elasticsearch documentation
The documentation is exhaustive, so will surely have solutions to most of your problems.

Ben Corlett’s Laracon Talk
This talk covers the basics of setting up and using Elasticsearch with Laravel.

Spatie’s Search Index Package
A package that exposes Elasticsearch and Algolia. I used this to see how other developers dealt with search indexes.

Integrating Elasticsearch – Madewithlove
A great summary of using Elasticsearch with a Laravel 4.2 application

Elasticquent
Maps Laravel Models to Elasticsearch types

14 comments

  1. Great post Mike!

    I’ve been using Elasticsearch with a Laravel 4.2 app for the last 6+ months now. It’s been a blast! There’s so much to learn when it comes to indexing and searching, like setting up mappings, stemming, advanced filtering/querying and all that. Maybe I’ll take some time to write about it too.

  2. Thanks for the post! Do you or anyone happen to know what the reccomended RAM would be required to run Elastic Search on a Forge / Digital Ocean box for a small to medium Laravel app that also hosts MySQL, redis, and a node process all on the same box?

    1. Thanks for reading! According to the documentation (https://www.elastic.co/guide/en/elasticsearch/guide/master/hardware.html) they recommend between 32GB and 64GB of RAM. I was able to get by with less (8GB) for a particular use case where I could limit the amount of data indexed, and with some tweaking (limiting shards and putting a cap on the memory granted to elasticsearch).

      You’d need to try it out on a non-production environment because elasticsearch can easily bring your server down if it’s hogging all the memory!

  3. nice tutorial, but someone who is new to laravel don´t know where to put these snippets into which files, because you didn´t explain that ;-(

    1. You’re right! This article is aimed at Laravel developers who are new to Elasticsearch. Having said that, the only code that needs to be placed in a particular location are the “register” and “boot” function snippets that need to go into a service provider (https://laravel.com/docs/providers).

      Feel free to DM me @michaelstivala with questions.

      1. many thanks for the info. i learned to love laravel the last few weekks :-)) so most things are clear, but some concepts, like hooks, special providers… not 😉 because i didn´t need them till now, so when i see tutorials like this, its really important to know where to put what, to make this running, even if you don´t understand everything 100% – but thats the magic i guess everyone wants to find out 😉

  4. [ErrorException]
    Argument 1 passed to AppElasticElastic::__construct() must be an instance of ElasticsearchClient, none given when I tried to initialize the Elastic class.

Leave a comment

Your email address will not be published.