Angular 6 ElasticSearch example – Documents Pagination with Scroll

In the post, we had known how to get All Documents in Index. This Angular 6 Elasticsearch tutorial shows you way to do pagination with scroll.

Next: Angular 6 ElasticSearch example – simple Full Text Search

Elasticsearch Tutorials:
Elasticsearch Overview
ElasticSearch Filter vs Query
ElasticSearch Full Text Queries – Basic

Elasticsearch Documents Pagination with Scroll

Add ElasticSearch to Angular 6 Project

Please visit Angular 6 ElasticSearch – Quick Start – How to add Elasticsearch.js for details.

With the post, you will know how to:
– Add ElasticSearch to Angular 6 Project
– Use it in the project

Add Document & Get All Documents in Index

– Add Document:
>> Please visit Angular 6 ElasticSearch example – Add Document to Index.

angular-6-elasticsearch-example-add-document-to-index-add-document

– Get All Documents:
>> Please visit Angular 6 ElasticSearch example – Get All Documents in Index.

angular-6-elasticsearch-example-show-documents-in-index-result

Get All Documents in Index with Scroll
Search with Scroll Param

In the Client.search() function, we add a SearchParam: scroll, set size for number of items per page and sort by id:

import { Client } from 'elasticsearch-browser';

@Injectable()
export class ElasticsearchService {

  private client: Client;

  getAllDocumentsWithScroll(_index, _type, _size): any {
    return this.client.search({
      index: _index,
      type: _type,
      // Set to 1 minute because we are calling right back
      // (Elasticsearch keeps the search context open for another 1m)
      scroll: '1m',
      filterPath: ['hits.hits._source', 'hits.total', '_scroll_id'],
      body: {
        'size': _size,
        'query': {
          'match_all': {}
        },
        'sort': [
          { '_uid': { 'order': 'asc' } }
        ]
      }
    });
  }
}

Assume that _size is 3, the response of search() function should be like:

TRACE: 2018-08-16T10:07:49Z
  -> POST http://localhost:9200/gkz_index/customer/_search?scroll=1m&filter_path=hits.hits._source%2Chits.total%2C_scroll_id
  {
    "size": 3,
    "query": {
      "match_all": {}
    },
    "sort": [
      {
        "_uid": {
          "order": "asc"
        }
      }
    ]
  }
  <- 200
  {
    "_scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAABFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAAAhZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAAAMWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQ==",
    "hits": {
      "total": 8,
      "hits": [
        {
          "_source": {
            "fullname": "David Louis",
            "age": 29,
            "address": "P.O. Box 399 4275 Amet Street\nWest Allis NC 36734",
            "published": "15/08/2018, 10:32:23"
          }
        },
        {
          "_source": {
            "fullname": "Jamie Konan",
            "age": 33,
            "address": "Ap #443-336 Ullamcorper. Street\nVisalia VA 54886",
            "published": "15/08/2018, 10:32:47"
          }
        },
        {
          "_source": {
            "fullname": "Katherin Kohen",
            "age": 22,
            "address": "1964 Facilisis Avenue\nBell Gardens Texas 87065",
            "published": "15/08/2018, 10:33:09"
          }
        }
      ]
    }
  }

We will use _scroll_id to get documents in next page.

Scroll to next page
  // ...

  getNextPage(scroll_id): any {
    return this.client.scroll({
      scrollId: scroll_id,
      scroll: '1m',
      filterPath: ['hits.hits._source', 'hits.total', '_scroll_id']
    });
  }

The response of search() function will contain next 3 (or less than 3) documents and should be like:

TRACE: 2018-08-16T10:08:43Z
  -> POST http://localhost:9200/_search/scroll?scroll=1m&filter_path=hits.hits._source%2Chits.total%2C_scroll_id
  {
    "scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAABFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAAAhZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAAAMWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQ=="
  }
  <- 200
  {
    "_scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAABFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAAAhZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAAAMWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQ==",
    "hits": {
      "total": 8,
      "hits": [
        {
          "_source": {
            "fullname": "Keegan Blair",
            "age": 38,
            "address": "Ap #761-2515 Egestas. Rd.\nManitowoc TN 07528",
            "published": "16/08/2018, 10:04:16"
          }
        },
        {
          "_source": {
            "fullname": "Katelyn Cooper",
            "age": 32,
            "address": "6059 Sollicitudin Road\nBurlingame Colorado 26278",
            "published": "16/08/2018, 10:07:07"
          }
        },
        {
          "_source": {
            "fullname": "Jack Smith",
            "age": 25,
            "address": "P.O. Box 902 3472 Ullamcorper Street\nLynchburg DC 29738",
            "published": "16/08/2018, 10:09:54"
          }
        }
      ]
    }
  }

Practice

0. Overview

Goal:

angular-6-elasticsearch-example-pagination-result

>> Click Next button for next page.

1. Generate Service & Components

Run cmd:
ng g s elasticsearch
ng g c customer/add-customer
ng g c customer/customer-details
ng g c customer/show-customers

=> Project Structure:

angular-6-elasticsearch-example-pagination-scroll-project-structure.png

2. App Module

app.module.ts

import { BrowserModule } from '@angular/platform-browser';
import { NgModule } from '@angular/core';
import { FormsModule, ReactiveFormsModule } from '@angular/forms';
import { AppRoutingModule } from './app-routing.module';

import { AppComponent } from './app.component';
import { AddCustomerComponent } from './customer/add-customer/add-customer.component';
import { CustomerDetailsComponent } from './customer/customer-details/customer-details.component';
import { ShowCustomersComponent } from './customer/show-customers/show-customers.component';

@NgModule({
  declarations: [
    AppComponent,
    AddCustomerComponent,
    CustomerDetailsComponent,
    ShowCustomersComponent
  ],
  imports: [
    BrowserModule,
    FormsModule,
    ReactiveFormsModule,
    AppRoutingModule
  ],
  providers: [],
  bootstrap: [AppComponent]
})
export class AppModule { }
3. ElasticSearch Service

elasticsearch.service.ts

import { Injectable } from '@angular/core';
import { Client } from 'elasticsearch-browser';

@Injectable({
  providedIn: 'root'
})
export class ElasticsearchService {

  private client: Client;

  private queryalldocs = {
    'query': {
      'match_all': {}
    }
  };

  constructor() {
    if (!this.client) {
      this.connect();
    }
  }

  private connect() {
    this.client = new Client({
      host: 'http://localhost:9200',
      log: 'trace'
    });
  }

  isAvailable(): any {
    return this.client.ping({
      requestTimeout: Infinity,
      body: 'hello grokonez!'
    });
  }

  addToIndex(value): any {
    return this.client.create(value);
  }

  getAllDocuments(_index, _type): any {
    return this.client.search({
      index: _index,
      type: _type,
      body: this.queryalldocs,
      filterPath: ['hits.hits._source']
    });
  }

  getAllDocumentsWithScroll(_index, _type, _size): any {
    return this.client.search({
      index: _index,
      type: _type,
      scroll: '1m',
      filterPath: ['hits.hits._source', 'hits.total', '_scroll_id'],
      body: {
        'size': _size,
        'query': {
          'match_all': {}
        },
        'sort': [
          { '_uid': { 'order': 'asc' } }
        ]
      }
    });
  }

  getNextPage(scroll_id): any {
    return this.client.scroll({
      scrollId: scroll_id,
      scroll: '1m',
      filterPath: ['hits.hits._source', 'hits.total', '_scroll_id']
    });
  }
}
4. Components

customer/customer.interface.ts

export interface Customer {
    fullname: string;
    age: number;
    address: string;
    published: string;
}

export interface CustomerSource {
    source: Customer;
}
4.1 Add Document Component

Please visit: 4_Add_Document_Component (in the previous post) for details.

4.2 Details Component

customer-details.component.ts

import { Component, OnInit, Input } from '@angular/core';
import { Customer } from '../customer.interface';

@Component({
  selector: 'customer-details',
  templateUrl: './customer-details.component.html',
  styleUrls: ['./customer-details.component.css']
})
export class CustomerDetailsComponent implements OnInit {

  @Input() customer: Customer;

  constructor() { }

  ngOnInit() {
  }

}

customer-details.component.html

{{customer.fullname}}
{{customer.age}}
{{customer.address}}
{{customer.published}}

4.3 Show Documents Component

show-customers.component.ts

import { Component, OnInit } from '@angular/core';
import { CustomerSource } from '../customer.interface';
import { ElasticsearchService } from '../../elasticsearch.service';

@Component({
  selector: 'show-customers',
  templateUrl: './show-customers.component.html',
  styleUrls: ['./show-customers.component.css']
})
export class ShowCustomersComponent implements OnInit {

  private static readonly INDEX = 'gkz_index';
  private static readonly TYPE = 'customer';
  private static readonly SIZE = 3;

  customerSources: CustomerSource[];
  haveNextPage = false;
  scrollID = '';
  notice = '';

  constructor(private es: ElasticsearchService) {
    this.scrollID = '';
    this.notice = '';
    this.haveNextPage = false;
  }

  ngOnInit() {
    this.es.getAllDocumentsWithScroll(
      ShowCustomersComponent.INDEX,
      ShowCustomersComponent.TYPE,
      ShowCustomersComponent.SIZE).then(
        response => {
          this.customerSources = response.hits.hits;

          if (response.hits.hits.length < response.hits.total) {
            this.haveNextPage = true;
            this.scrollID = response._scroll_id;
          }
          console.log(response);
        }, error => {
          console.error(error);
        }).then(() => {
          console.log('Show Customer Completed!');
        });
  }

  showNextPage() {
    this.es.getNextPage(this.scrollID).then(
      response => {
        this.customerSources = response.hits.hits;
        if (!response.hits.hits) {
          this.haveNextPage = false;
          this.notice = 'There are no more Customers!';
        }
        console.log(response);
      }, error => {
        console.error(error);
      }).then(() => {
        console.log('Show Customer Completed!');
      });
  }
}

show-customers.component.html

{{notice}}

age" class="btn btn-primary">Next
5. App Routing Module

app-routing.module.ts

import { AddCustomerComponent } from './customer/add-customer/add-customer.component';
import { ShowCustomersComponent } from './customer/show-customers/show-customers.component';

import { NgModule } from '@angular/core';
import { RouterModule, Routes } from '@angular/router';

const routes: Routes = [
    { path: '', redirectTo: 'add', pathMatch: 'full' },
    { path: 'add', component: AddCustomerComponent },
    { path: 'customers', component: ShowCustomersComponent }
];

@NgModule({
    imports: [RouterModule.forRoot(routes)],
    exports: [RouterModule]
})
export class AppRoutingModule { }
6. App Component

app.component.ts

import { Component } from '@angular/core';

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css']
})
export class AppComponent {
  title = 'grokonez';
  description = 'Angular - Elasticsearch';
}

app.component.html

{{title}}

{{description}}

7. Check Result

Run Angular 6 App, go to http://localhost:4200/, add Customer Data, then choose Customers tab:

angular-6-elasticsearch-example-pagination-result

Click Next button several times to view more documents:

angular-6-elasticsearch-example-pagination-result-new-page

Open Browser Console, you can see:

TRACE: 2018-08-16T10:16:27Z
  -> POST http://localhost:9200/gkz_index/customer/_search?scroll=1m&filter_path=hits.hits._source%2Chits.total%2C_scroll_id
  {
    "size": 3,
    "query": {
      "match_all": {}
    },
    "sort": [
      {
        "_uid": {
          "order": "asc"
        }
      }
    ]
  }
  <- 200
  {
    "_scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAATFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAAFBZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAABUWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQ==",
    "hits": {
      "total": 8,
      "hits": [
        {
          "_source": {
            // customer-1
          }
        },
        {
          "_source": {
            // customer-2
          }
        },
        {
          "_source": {
            customer-3
          }
        }
      ]
    }
  }
------------------------
Show Customer Completed!
TRACE: 2018-08-16T10:16:52Z
  -> POST http://localhost:9200/_search/scroll?scroll=1m&filter_path=hits.hits._source%2Chits.total%2C_scroll_id
  {
    "scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAATFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAAFBZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAABUWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQ=="
  }
  <- 200
  {
    "_scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAATFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAAFBZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAABUWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQ==",
    "hits": {
      "total": 8,
      "hits": [
        {
          "_source": {
            // customer-4
          }
        },
        {
          "_source": {
            // customer-5
          }
        },
        {
          "_source": {
            // customer-6
          }
        }
      ]
    }
  }
------------------------
Show Customer Completed!
TRACE: 2018-08-16T10:16:54Z
  -> POST http://localhost:9200/_search/scroll?scroll=1m&filter_path=hits.hits._source%2Chits.total%2C_scroll_id
  {
    "scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAATFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAAFBZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAABUWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQ=="
  }
  <- 200
  {
    "_scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAATFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAAFBZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAABUWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQ==",
    "hits": {
      "total": 8,
      "hits": [
        {
          "_source": {
            // customer-7
          }
        },
        {
          "_source": {
            // customer-8
          }
        }
      ]
    }
  }

When there is no more document:

TRACE: 2018-08-16T10:21:09Z
  -> POST http://localhost:9200/_search/scroll?scroll=1m&filter_path=hits.hits._source%2Chits.total%2C_scroll_id
  {
    "scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAATFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAAFBZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAABUWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQ=="
  }
  <- 200
  {
    "_scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAATFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAAFBZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAABUWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQ==",
    "hits": {
      "total": 8
    }
  }

Sourcecode

Angular6Elasticsearch-pagination-scroll



By grokonez | August 16, 2018.


Related Posts


1 thought on “Angular 6 ElasticSearch example – Documents Pagination with Scroll”

Got Something To Say:

Your email address will not be published. Required fields are marked *

*