Angular 4 ElasticSearch example – Documents Pagination with Scroll

In the post, we had known how to get All Documents in Index. This tutorial shows you way to do pagination with scroll function.

Related Post:
Angular 4 ElasticSearch – Quick Start – How to add Elasticsearch.js
Angular 4 ElasticSearch example – How to create an Index
Angular 4 ElasticSearch example – Add Document to Index
Angular 4 ElasticSearch example – Get All Documents in Index
Angular 4 ElasticSearch example – simple Full Text Search

Elasticsearch Tutorials:
Elasticsearch Overview
ElasticSearch Filter vs Query
ElasticSearch Full Text Queries – Basic

I. How to

1. Add ElasticSearch to Angular 4 Project

Please visit Angular 4 ElasticSearch – Quick Start – How to add Elasticsearch.js for details.

With the post, you will know how to:
– Install ElasticSearch
– Add ElasticSearch to Angular 4 Project
– Use it in the project

2. Create Index & Check Connection

Please visit Angular 4 ElasticSearch example – How to create an Index for details.

3. Get All Documents in Index with Scroll
3.1 Search with Scroll Param

In the Client.search() function, we add a SearchParam: scroll, set size for number of items per page and sort by id:

import { Client } from 'elasticsearch';

@Injectable()
export class ElasticsearchService {

  private client: Client;

  getAllDocumentsWithScroll(_index, _type, _size): any {
    return this.client.search({
      index: _index,
      type: _type,
      // Set to 1 minute because we are calling right back
      // (Elasticsearch keeps the search context open for another 1m)
      scroll: '1m',
      filterPath: ['hits.hits._source', 'hits.total', '_scroll_id'],
      body: {
        'size': _size,
        'query': {
          'match_all': {}
        },
        'sort': [
          { '_uid': { 'order': 'asc' } }
        ]
      }
    });
  }
}

Assume that _size is 3, the response of search() function should be like:

{
  "_scroll_id": "DnF1ZXJ5VGhlbkZldGNoBQAAAAAAABopFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAaKhZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAGisWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQAAAAAAABosFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAaLRZnTUwtc2U0OFRuMmdKRnNoVVVfdmRR",
  "hits": {
    "total": 7,
    "hits": [
      {
        "_source": {
          "fullname": "Jack Smith",
          "age": 25,
          "address": "P.O. Box 902 3472 Ullamcorper Street\nLynchburg DC 29738",
          "published": "14/10/2017, 20:47:34"
        }
      },
      {
        "_source": {
          "fullname": "Jamie Konan",
          "age": 33,
          "address": "Ap #443-336 Ullamcorper. Street\nVisalia VA 54886",
          "published": "14/10/2017, 20:47:58"
        }
      },
      {
        "_source": {
          "fullname": "David James",
          "age": 36,
          "address": "737-2580 At Street\nIndependence Texas 87535",
          "published": "14/10/2017, 20:48:39"
        }
      }
    ]
  }
}

We will use _scroll_id to get documents in next page.

3.2 Scroll to next page
  // ...

  getNextPage(scroll_id): any {
    return this.client.scroll({
      scrollId: scroll_id,
      scroll: '1m',
      filterPath: ['hits.hits._source', 'hits.total', '_scroll_id']
    });
  }

The response of search() function will contain next 3 (or less than 3) documents and should be like:

{
  "_scroll_id": "DnF1ZXJ5VGhlbkZldGNoBQAAAAAAABopFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAaKhZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAGisWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQAAAAAAABosFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAaLRZnTUwtc2U0OFRuMmdKRnNoVVVfdmRR",
  "hits": {
    "total": 7,
    "hits": [
      {
        "_source": {
          "fullname": "Katherin Kohen",
          "age": 22,
          "address": "1964 Facilisis Avenue\nBell Gardens Texas 87065",
          "published": "14/10/2017, 20:49:09"
        }
      },
      {
        "_source": {
          "fullname": "David Louis",
          "age": 29,
          "address": "P.O. Box 399 4275 Amet Street\nWest Allis NC 36734",
          "published": "14/10/2017, 20:49:33"
        }
      },
      {
        "_source": {
          "fullname": "Jasper Carney",
          "age": 35,
          "address": "1195 Lobortis Rd.\nNew Orleans New Hampshire 71983",
          "published": "15/10/2017, 00:07:54"
        }
      }
    ]
  }
}

II. Practice

0. Overview

Goal:
angular-4-elasticsearch-pagination-scroll-overview

Project Structure:
angular-4-elasticsearch-get-all-documents-structure

1. App Module
import { NgModule } from '@angular/core';
import { BrowserModule } from '@angular/platform-browser';
import { FormsModule, ReactiveFormsModule } from '@angular/forms';
import { AppRoutingModule } from './app-routing.module';

import { AppComponent } from './app.component';
import { ElasticsearchService } from './elasticsearch.service';
import { AddCustomerComponent } from './customer/add-customer/add-customer.component';
import { ShowCustomersComponent } from './customer/show-customers/show-customers.component';
import { CustomerDetailsComponent } from './customer/customer-details/customer-details.component';

@NgModule({
  declarations: [
    AppComponent,
    AddCustomerComponent,
    ShowCustomersComponent,
    CustomerDetailsComponent
  ],
  imports: [
    BrowserModule,
    FormsModule,
    ReactiveFormsModule,
    AppRoutingModule
  ],
  providers: [ElasticsearchService],
  bootstrap: [AppComponent]
})

export class AppModule { }
2. ElasticSearch Service
import { Injectable } from '@angular/core';

import { Client } from 'elasticsearch';

@Injectable()
export class ElasticsearchService {

  private client: Client;

  constructor() {
    if (!this.client) {
      this.connect();
    }
  }

  private connect() {
    this.client = new Client({
      host: 'http://localhost:9200',
      log: 'trace'
    });
  }

  createIndex(name): any {
    return this.client.indices.create(name);
  }

  isAvailable(): any {
    return this.client.ping({
      requestTimeout: Infinity,
      body: 'hello JavaSampleApproach!'
    });
  }

  addToIndex(value): any {
    return this.client.create(value);
  }

  getAllDocumentsWithScroll(_index, _type, _size): any {
    return this.client.search({
      index: _index,
      type: _type,
      scroll: '1m',
      filterPath: ['hits.hits._source', 'hits.total', '_scroll_id'],
      body: {
        'size': _size,
        'query': {
          'match_all': {}
        },
        'sort': [
          { '_uid': { 'order': 'asc' } }
        ]
      }
    });
  }

  getNextPage(scroll_id): any {
    return this.client.scroll({
      scrollId: scroll_id,
      scroll: '1m',
      filterPath: ['hits.hits._source', 'hits.total', '_scroll_id']
    });
  }
}
3. Components
3.1 Add Document Component

Please visit: 3_Add_Document_Component for details.

3.2 Details Component

You can find it at: 3.2_Details_Component (in the previous post).

3.3 Show Documents Component

customer.interface.ts

export interface Customer {
    fullname: string;
    age: number;
    address: string;
    published: string;
}

export interface CustomerSource {
    source: Customer;
}

show-customers.component.ts

import { Component, OnInit } from '@angular/core';
import { Observable } from 'rxjs/Observable';

import { CustomerSource } from '../customer.interface';
import { ElasticsearchService } from '../../elasticsearch.service';

@Component({
  selector: 'show-customers',
  templateUrl: './show-customers.component.html',
  styleUrls: ['./show-customers.component.css']
})
export class ShowCustomersComponent implements OnInit {

  private static readonly INDEX = 'jsa_customer_idx';
  private static readonly TYPE = 'customer';
  private static readonly SIZE = 3;

  customerSources: CustomerSource[];
  haveNextPage = false;
  scrollID = '';
  notice = '';

  constructor(private es: ElasticsearchService) {
    this.scrollID = '';
    this.notice = '';
    this.haveNextPage = false;
  }

  ngOnInit() {
    this.es.getAllDocumentsWithScroll(
      ShowCustomersComponent.INDEX,
      ShowCustomersComponent.TYPE,
      ShowCustomersComponent.SIZE).then(
      response => {
        this.customerSources = response.hits.hits;

        if (response.hits.hits.length < response.hits.total) {
          this.haveNextPage = true;
          this.scrollID = response._scroll_id;
        }
        console.log(response);
      }, error => {
        console.error(error);
      }).then(() => {
        console.log('Show Customer Completed!');
      });
  }

  showNextPage() {
    this.es.getNextPage(this.scrollID).then(
      response => {
        this.customerSources = response.hits.hits;
        if (!response.hits.hits) {
          this.haveNextPage = false;
          this.notice = 'There are no more Customers!';
        }
        console.log(response);
      }, error => {
        console.error(error);
      }).then(() => {
        console.log('Show Customer Completed!');
      });
  }
}

show-customers.component.html

{{notice}}

4. App Routing Module
import { AddCustomerComponent } from './customer/add-customer/add-customer.component';
import { ShowCustomersComponent } from './customer/show-customers/show-customers.component';

import { NgModule } from '@angular/core';
import { RouterModule, Routes } from '@angular/router';

const routes: Routes = [
    { path: '', redirectTo: 'add', pathMatch: 'full' },
    { path: 'add', component: AddCustomerComponent },
    { path: 'customers', component: ShowCustomersComponent }
];

@NgModule({
    imports: [RouterModule.forRoot(routes)],
    exports: [RouterModule]
})

export class AppRoutingModule { }
5. App Component

app.component.ts

import {Component} from '@angular/core';

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css']
})
export class AppComponent {
  title = 'JavaSampleApproach';
  description = 'Angular4-SpringBoot Demo';
}

app.component.html

{{title}}

{{description}}

6. Check Result

Run Angular 4 App, go to http://localhost:4200/, add Customer Data, then choose Customers tab:
angular-4-elasticsearch-pagination-scroll-overview

Click Next button several times to view more documents:
angular-4-elasticsearch-pagination-scroll-result-next

Open Browser Console, you can see:

TRACE: 2017-10-16T11:05:42Z
  -> POST http://localhost:9200/jsa_customer_idx/customer/_search?scroll=1m&filter_path=hits.hits._source%2Chits.total%2C_scroll_id
  {
    "size": 3,
    "query": {
      "match_all": {}
    },
    "sort": [
      {
        "_uid": {
          "order": "asc"
        }
      }
    ]
  }
  <- 200
  {
    "_scroll_id": "DnF1ZXJ5VGhlbkZldGNoBQAAAAAAAAFOFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAABTxZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAAVAWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQAAAAAAAAFRFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAABUhZnTUwtc2U0OFRuMmdKRnNoVVVfdmRR",
    "hits": {
      "total": 7,
      "hits": [
        {
          "_source": {
            // customer-1
          }
        },
        {
          "_source": {
            // customer-2
          }
        },
        {
          "_source": {
            // customer-3
          }
        }
      ]
    }
  }
------------------------
Show Customer Completed!
TRACE: 2017-10-16T11:05:44Z
  -> POST http://localhost:9200/_search/scroll?scroll=1m&filter_path=hits.hits._source%2Chits.total%2C_scroll_id
  {
    "scroll_id": "DnF1ZXJ5VGhlbkZldGNoBQAAAAAAAAFOFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAABTxZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAAVAWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQAAAAAAAAFRFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAABUhZnTUwtc2U0OFRuMmdKRnNoVVVfdmRR"
  }
  <- 200
  {
    "_scroll_id": "DnF1ZXJ5VGhlbkZldGNoBQAAAAAAAAFOFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAABTxZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAAVAWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQAAAAAAAAFRFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAABUhZnTUwtc2U0OFRuMmdKRnNoVVVfdmRR",
    "hits": {
      "total": 7,
      "hits": [
        {
          "_source": {
            // customer-4
          }
        },
        {
          "_source": {
            // customer-5
          }
        },
        {
          "_source": {
            // customer-6
          }
        }
      ]
    }
  }
------------------------
Show Customer Completed!
TRACE: 2017-10-16T11:06:46Z
  -> POST http://localhost:9200/_search/scroll?scroll=1m&filter_path=hits.hits._source%2Chits.total%2C_scroll_id
  {
    "scroll_id": "DnF1ZXJ5VGhlbkZldGNoBQAAAAAAAAFOFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAABTxZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAAVAWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQAAAAAAAAFRFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAABUhZnTUwtc2U0OFRuMmdKRnNoVVVfdmRR"
  }
  <- 200
  {
    "_scroll_id": "DnF1ZXJ5VGhlbkZldGNoBQAAAAAAAAFOFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAABTxZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAAVAWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQAAAAAAAAFRFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAABUhZnTUwtc2U0OFRuMmdKRnNoVVVfdmRR",
    "hits": {
      "total": 7,
      "hits": [
        {
          "_source": {
            // last-customer
          }
        }
      ]
    }
  }

When there is no more document:

Show Customer Completed!
TRACE: 2017-10-16T11:08:17Z
  -> POST http://localhost:9200/_search/scroll?scroll=1m&filter_path=hits.hits._source%2Chits.total%2C_scroll_id
  {
    "scroll_id": "DnF1ZXJ5VGhlbkZldGNoBQAAAAAAAAFOFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAABTxZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAAVAWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQAAAAAAAAFRFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAABUhZnTUwtc2U0OFRuMmdKRnNoVVVfdmRR"
  }
  <- 200
  {
    "_scroll_id": "DnF1ZXJ5VGhlbkZldGNoBQAAAAAAAAFOFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAABTxZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAAVAWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQAAAAAAAAFRFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAABUhZnTUwtc2U0OFRuMmdKRnNoVVVfdmRR",
    "hits": {
      "total": 7
    }
  }

III. Sourcecode

Angular4ElasticSearch-pagination-scroll



By grokonez | October 16, 2017.

Last updated on August 16, 2018.



Related Posts


1 thought on “Angular 4 ElasticSearch example – Documents Pagination with Scroll”

Got Something To Say:

Your email address will not be published. Required fields are marked *

*