Decompose the result of a MySql query that is in JSON to new indexable fields in elasticsearch with logstash

Question

Decompose the result of a MySql query that is in JSON to new indexable fields in elasticsearch with logstash

Navigation

#1 by (0 votes)

1

I want to put the result of a query from mysql to elasticsearch through logstash, I have a field called Detail that contains a string that is a JSON, which can contain the following:

{
  "interna": true,
  "ContactoNombre": "Marvel",
  "ContactoCorreo": "[email protected]",
  "Salario": "900 - 1000"
}

How can I obtain and separate this json in the following fields:

salariomin: 900
salariomax: 1000
rangosalario 9000 - 1000
internal: true or 1

In the configuration of logstash with filters, I have been trying with the json plugin in the following way:

filter{
    json {
        source => "detalle"
        remove_field => "detalle"
        target => "Salario"
        add_field=> { "interna"=> "Salario[0]" }
    }
}

The result in elasticsearch of this is:

"Salario": {
     "interna": true,
      "ContactoNombre": "Marvel ",
      "ContactoCorreo": "[email protected]",
      "Salario": "900 - 1000"
},

I have also seen the split plugin, but I really can not understand how to decompose the salary field into two fields.

I thank you in advance for your help.

PD. The labels logstash and elasticsearch do not exist in the Stackoverflow

mysql

asked by Peter Rojas 28.12.2016 в 16:49

source

1 answer

Node ordering? [closed] Disable USB ports from CMD or Powershell

score 0 · Answer 1

At the end my logstash file was like this:

filter{
    json {
        source => "detalle"
        remove_field => "detalle"
        target => "jsondetalle"
    }
    if "interna" in [jsondetalle]{
        mutate{
            add_field => {
               "interna" => "%{[jsondetalle][interna]}"
               "rangosalario" => "%{[jsondetalle][Salario]}"
            }
          #  remove_field => "jsondetalle"
        }
        grok {
            match => { "rangosalario" => "%{NUMBER:salarymin:float} - %{NUMBER:salarymax:float}"}
            remove_field => "rangosalario"
        }
    }
}