...
For each of the operations you'll need your Logsene application token when calling logsene-receiver.sematext.com. In the following examples we will use a "dummy token" - cc5e9c1b-3046-4e43-998e-2a0b2c01b912 as the token. You should use your real Logsene App token, of course.
URI based search
The most simple simplest search method to get your data out of Logsene is fully compatible with URI Search in Elasticsearch (https://www.elastic.co/guide/en/elasticsearch/reference/current/search-uri-request.html). You need to provide the query using the q parameter. For example, to search for the internal and connection terms you could would run the following command:
| No Format |
|---|
curl -XGET 'logsene-receiver.sematext.com/cc5e9c1b-3046-4e43-998e-2a0b2c01b912/_search?pretty&q=+internal%20+connection' |
Note: To learn more about Apache Lucene query syntaxtsyntax, please refer to https://lucene.apache.org/core/5_2_1/queryparser/org/apache/lucene/queryparser/classic/package-summary.html
...
The request body based search allows lets us to leverage full Elasticsearch query DSL language (https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-queries.html) along with the its filtering capabilities (https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-filters.html) and aggregations (https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations.html). With full featured Elasticsearch query API we can search and find any document data we are really looking for.
For example, to find documents that match the internal and connection terms you could run the following query:
| No Format |
|---|
curl -XGET 'logsene-receiver.sematext.com/cc5e9c1b-3046-4e43-998e-2a0b2c01b912/_search?pretty' -d '{
"query" : {
"bool" : {
"must" : [
{
"match" : {
"_all" : "internal"
}
},
{
"match" : {
"_all" : "connection"
}
}
]
}
}
}' |
If we would now want to analyze the data, To analyze this data further we can add aggregations to our query (https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations.html) to find common status responses for example:
...
Logsene, just like Elasticsearch, talks to you using JSON. This is very convenient as JSON is readable by machines and humans. The response of Elasticsearch looks as followsHere's an example response:
| No Format |
|---|
{
"took" : 10,
"timed_out" : false,
"_shards" : {
"total" : 3,
"successful" : 3,
"failed" : 0
},
"hits" : {
"total" : 126149,
"max_score" : 0.57406324,
"hits" : [ {
...
]
},
"aggregations" : {
...
}
} |
As you can see the response is a JSON object , which contains with three main sections:
- the header, which gives us information about the status of the response, like time it took to render it, if the query was timed out,
- the hits object that includes information about returned results (total count and maximum score) and of course the hits array, which
...
- includes the returned documents (10 by default),
- the aggregations object that includes aggregations results if we've used aggregations in our query
...
The real example of the results returned look as follows:
...
The real time GET operation is very simple and it allows lets us to get a single document out of a particular Logsene index. To retrieve a document we need to provide Logsene with the following infomationinformation:
- index name - it will be <token>_free if you are not during application trial period your Logsene app trial has expired and you don't have an active subscription a paid plan, or <token>_<date> (where date is YYYY-MM-DD) when you have a paid subscription plan for the Logsene service,
- type name - the type of the document you want to retrieve,
- document identifier - the identifier of the document.
For example, to retrieve a document with identifier AU29tJz0UV2O9bWZ_KkU and type apache from our example application identified by cc5e9c1b-3046-4e43-998e-2a0b2c01b912 token (application is free, which means that we need to append the token with _free postfix to get the index name) we would run the following command:
...
In addition to supporting the real time GET functionality, Logsene allows user to lets one leverage Elasticsearch MGet API (https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-multi-get.html), which allows us to retrieve multiple document using the real time GET API in a single request. For example, to retrieve documents with identifier AU29tJz0UV2O9bWZ_KkU and AU29rlOPUV2O9bWZ-Daw which are of type apache from our example application identified by cc5e9c1b-3046-4e43-998e-2a0b2c01b912 token we would run the following request:
...
As you can see, we are sending a HTTP GET request to the _mget REST end-point of Logsene receiver and we provide an get back a JSON object that contains the docs array. Each entry of the docs array is identifying a single document by providing the index name (the _index property), the type name (the _type property) and the document identifier (the _id property).
...
Multiple Search operations in a single request
Similar to MGet Logsene offers the possibility to , Logsene lets you run multiple search requests in a single HTTP request using Elasticsearch Multi Search API (https://www.elastic.co/guide/en/elasticsearch/reference/current/search-multi-search.html). The request needs to be run against _msearch REST end-point and each query needs to include two lines - meta line defining the index name and the second a line defining the query using Elasticsearch query DSL (https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-queries.html).
...
| No Format |
|---|
curl -XGET 'logsene-receiver.sematext.com/_msearch?pretty' --data-binary '{ "index" : "cc5e9c1b-3046-4e43-998e-2a0b2c01b912_free" }
{ "query" : { "match_all" : {} }, "size" : 1 }
{ "index" : "cc5e9c1b-3046-4e43-998e-2a0b2c01b912_free" }
{ "query" : { "term" : { "status" : 200 } }, "size" : 1 }' |
One thing to remember about the Keep in mind that Multiple Search API is using --data-binary switch in the curl command to keep the new line characters in the request. This is crucial to make the Multiple Search API working correctly.
...