TransportError: TransportError(503, 'search_phase_execution_exception') - python

I'm trying to read documents from an index using Python with Elasticsearch. I get an error I can't find the solution to. Does anyone know how I can fix this?
I've looked online for the same problem, but I can't find any related articles/solutions.
This is my code:
from elasticsearch import Elasticsearch
from pandasticsearch import Select
# Connection to ElasticSearch
es = Elasticsearch(['http://localhost:9200'],timeout=600)
# Retrieving all documents in index (no query given)
documents = es.search(index='myindex',body={"query": {"match_all": {}}})
# Convert the result to Pandas Dataframe
pandas_df = Select.from_dict(documents).to_pandas()
*Source: https://medium.com/#patni.sonal/elasticsearch-convert-your-index-data-into-pandas-dataframe-30c44ada2407
The error I get is:
GET http://localhost:9200/myindex/_search [status:503 request:0.062s]
GET http://localhost:9200/myindex/_search [status:503 request:0.027s]
GET http://localhost:9200/myindex/_search [status:503 request:0.030s]
GET http://localhost:9200/myindex/_search [status:503 request:0.017s]
Traceback (most recent call last):
File "<ipython-input-6-68e07be7c100>", line 1, in <module>
documents = es.search(index='myindex',body={"query": {"match_all": {}}})
File "C:\Users\XXXXXX\AppData\Local\Continuum\anaconda3\lib\site-
packages\elasticsearch\client\utils.py", line 76, in _wrapped
return func(*args, params=params, **kwargs)
File "C:\Users\XXXXXX\AppData\Local\Continuum\anaconda3\lib\site-
packages\elasticsearch\client\__init__.py", line 655, in search
doc_type, '_search'), params=params, body=body)
File "C:\Users\XXXXXX\AppData\Local\Continuum\anaconda3\lib\site-
packages\elasticsearch\transport.py", line 318, in perform_request
status, headers_response, data = connection.perform_request(method, url,
params, body, headers=headers, ignore=ignore, timeout=timeout)
File "C:\Users\XXXXXX\AppData\Local\Continuum\anaconda3\lib\site-
packages\elasticsearch\connection\http_urllib3.py", line 185, in
perform_request
self._raise_error(response.status, raw_data)
File "C:\Users\XXXXXX\AppData\Local\Continuum\anaconda3\lib\site-
packages\elasticsearch\connection\base.py", line 125, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code,
error_message, additional_info)
TransportError: TransportError(503, 'search_phase_execution_exception')
This is the elasticsearch log:
[2019-04-25T11:19:35,843][DEBUG][o.e.a.s.TransportSearchAction] [Cx6mYd6] All shards failed for phase: [query]
[2019-04-25T11:19:35,845][WARN ][r.suppressed ] [Cx6mYd6] path: /myindex/_search, params: {index=myindex}
org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed
at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:293) ~[elasticsearch-6.6.1.jar:6.6.1]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:133) ~[elasticsearch-6.6.1.jar:6.6.1]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:254) ~[elasticsearch-6.6.1.jar:6.6.1]
at org.elasticsearch.action.search.InitialSearchPhase.onShardFailure(InitialSearchPhase.java:101) ~[elasticsearch-6.6.1.jar:6.6.1]
at org.elasticsearch.action.search.InitialSearchPhase.lambda$performPhaseOnShard$1(InitialSearchPhase.java:209) ~[elasticsearch-6.6.1.jar:6.6.1]
at org.elasticsearch.action.search.InitialSearchPhase$1.doRun(InitialSearchPhase.java:188) [elasticsearch-6.6.1.jar:6.6.1]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:759) [elasticsearch-6.6.1.jar:6.6.1]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.6.1.jar:6.6.1]
at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) [elasticsearch-6.6.1.jar:6.6.1]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.6.1.jar:6.6.1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:1.8.0_181]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_181]
Does anyone has any idea on how to fix this?

Related

I made mistakes when I was using Python to insert simple data into elasticsearch

I just touched elastic search, and after some simple tutorials with python, there was an error inserting data. My elastic search version was 6.4.0, and my Python version was 3.6 macOS version was 10.13.6
code:
#from datetime import datetime
from elasticsearch import Elasticsearch
es = Elasticsearch()
es.indices.create(index='my-index', ignore=400)
es.index(index="my-index", doc_type="test-type", id=1, body={"any":"data01", "timestamp": datetime.now()})
Errors:
PUT /my-index/test-type/1 [status:406 request:0.002s]
Traceback (most recent call last):
File "/Users/kenneth/PycharmProjects/Python核心编程/Elasticsearch/Python对接Elasticsearch/demo.py", line 8, in <module>
es.index(index="my-index", doc_type="test-type", id=1, body={"any":"data01", "timestamp": datetime.now()})
File "/Users/kenneth/Library/Python/3.6/lib/python/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
return func(*args, params=params, **kwargs)
File "/Users/kenneth/Library/Python/3.6/lib/python/site-packages/elasticsearch/client/__init__.py", line 263, in index
_make_path(index, doc_type, id), params=params, body=body)
File "/Users/kenneth/Library/Python/3.6/lib/python/site-packages/elasticsearch/transport.py", line 307, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/Users/kenneth/Library/Python/3.6/lib/python/site-packages/elasticsearch/connection/http_urllib3.py", line 93, in perform_request
self._raise_error(response.status, raw_data)
File "/Users/kenneth/Library/Python/3.6/lib/python/site-packages/elasticsearch/connection/base.py", line 105, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.TransportError: TransportError(406, 'Content-Type header [] is not supported')

indexing json documents into Elasticsearch throws TransportError

I have a JSON file, which i'm loading using json.load method and then sending the content to Elasticsearch. But, it throws TransportError.
Sample code:
inputFile = sys.argv[1]
with open(inputFile) as jsonFile:
jsonData = json.load(jsonFile)
es = Elasticsearch([{'host':'x.x.x.175', 'port':9200}])
es.index(index="fmb_420", doc_type='fmb_420', body=jsonData)
Log error:
Traceback (most recent call last):
File "FMB_DB.py", line 12, in <module>
es.index(index="fmb_420", doc_type='fmb_420', body=jsonData)
File "/usr/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 76, in _wrapped
return func(*args, params=params, **kwargs)
File "/usr/lib/python2.7/site-packages/elasticsearch/client/__init__.py", line 319, in index
_make_path(index, doc_type, id), params=params, body=body)
File "/usr/lib/python2.7/site-packages/elasticsearch/transport.py", line 314, in perform_request
status, headers_response, data = connection.perform_request(method, url, params, body, headers=headers, ignore=ignore, timeout=timeout)
File "/usr/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 180, in perform_request
self._raise_error(response.status, raw_data)
File "/usr/lib/python2.7/site-packages/elasticsearch/connection/base.py", line 125, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.RequestError: TransportError(400, u'mapper_parsing_exception', u'not_x_content_exception: Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes')
Update:
I fixed this problem by adding root element in the JSON file, before the JSON was not having the root element.
I fixed this problem by adding the root element in JSON file.
for example from this to
{
"name":"alex",
"age":34
}
this:
{
"employee": {
"name":"alex",
"age":34
}
}

helpers.scan() doesn't scroll - ElasticSearch

I am unable to scan whole ElasticSearch index.
ElasticSearch version: 1.7.5
I am trying to scan index but it always fail after 1000 documents fetched.
res = helpers.scan(es, index="twitter", query={"query": {"match_all": {}}},
request_timeout=60)
for hit in res:
do_some_stuff()
Traceback:
WARNING:elasticsearch:GET http://example.com:9200/_search/scroll?scroll=5m [status:400 request:0.002s]
WARNING:elasticsearch:DELETE http://example.com:9200/_search/scroll [status:400 request:0.001s]
Traceback (most recent call last):
File "/project/.env/lib/python3.5/site-packages/elasticsearch/helpers/__init__.py", line 301, in scan
resp = client.scroll(scroll_id, scroll=scroll, request_timeout=request_timeout)
File "/project/.env/lib/python3.5/site-packages/elasticsearch/client/utils.py", line 73, in _wrapped
return func(*args, params=params, **kwargs)
File "/project/.env/lib/python3.5/site-packages/elasticsearch/client/__init__.py", line 1024, in scroll
params=params, body=body)
File "/project/.env/lib/python3.5/site-packages/elasticsearch/transport.py", line 312, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/project/.env/lib/python3.5/site-packages/elasticsearch/connection/http_urllib3.py", line 128, in perform_request
self._raise_error(response.status, raw_data)
File "/project/.env/lib/python3.5/site-packages/elasticsearch/connection/base.py", line 125, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.RequestError: <exception str() failed>
When I tried manually send request:
POST example.com:9200/twitter/_search?scroll=1m
{
"size": 100,
"query": {
"match_all": {}
}
}
It returned me 100 documents and scroll_id. But this scroll_id was 524 character long. Probably this can be a problem.
I have installed newestest elastic package client. So I downgraded from 6.1.1 to 1.7.0
ElasticSearch in version 1.7.0 doesn't support JSON in POST request for scroll id but newer versions support it.

Elasticsearch Missing Authentication token for REST request

I am working on using an elasticsearch database to store data I am pulling from online, however when I try to index the data in the database I receive an error.
Here is my code for creating and the indexing the data:
es = Elasticsearch()
es.index(index='weather', doc_type='data', body=doc)
However when I run this program, the second of those lines causes an error, here is the complete traceback:
Traceback (most recent call last):
File "weatherScraper.py", line 79, in <module>
main()
File "weatherScraper.py", line 73, in main
es.index(index='weather', doc_type='data', body=doc)
File "/home/alec/.local/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 73, in _wrapped
return func(*args, params=params, **kwargs)
File "/home/alec/.local/lib/python2.7/site-packages/elasticsearch/client/__init__.py", line 298, in index
_make_path(index, doc_type, id), params=params, body=body)
File "/home/alec/.local/lib/python2.7/site-packages/elasticsearch/transport.py", line 312, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/home/alec/.local/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 128, in perform_request
self._raise_error(response.status, raw_data)
File "/home/alec/.local/lib/python2.7/site-packages/elasticsearch/connection/base.py", line 125, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.AuthenticationException: TransportError(401, u'security_exception', u'missing authentication token for REST request [/weather/data]')
''missing authentication token' means you need to authenticate before you can talk to this Elasticsearch instance. To index documents, the user must have write access. You can include a username and password in a URL like this: http://user:password#hostname:post
For example, in a shell:
export ES_ENDPOINT="http://usernameWithWriteAccess:password#localhost:9200"
Then in python:
es = Elasticsearch(os.environ['ES_ENDPOINT'])

Django/Haystack error: elasticsearch.exceptions.RequestError: TransportError(400, 'parsing_exception',…)

I am using elasticsearch as backend for haystack in my Django project. I created everything required for a search as mentioned here. But when i search i throws a traceback error with TransportError(400, 'parsing_exception', 'no [query] registered for [filtered]').
I have googled for this issue. But don't get any solution. I would appreciate helping me solve this.
My traceback:
Traceback (most recent call last):
File "c:\python34\lib\site- packages\haystack\backends\elasticsearch_backend.py", line 524, in search
_source=True)
File "c:\python34\lib\site-packages\elasticsearch\client\utils.py", line 71, in _wrapped
return func(*args, params=params, **kwargs)
File "c:\python34\lib\site-packages\elasticsearch\client\__init__.py", line 569, in search
doc_type, '_search'), params=params, body=body)
File "c:\python34\lib\site-packages\elasticsearch\transport.py", line 327, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "c:\python34\lib\site-packages\elasticsearch\connection\http_urllib3.py", line 124, in perform_request
self._raise_error(response.status, raw_data)
File "c:\python34\lib\site-packages\elasticsearch\connection\base.py", line 122, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.RequestError: TransportError(400, 'parsing_exception', 'no [query] registered for [filtered]')
[28/Dec/2016 17:06:58]"GET /search/?q=code HTTP/1.1" 200 395
Update-1 : TraceBack after downgraded to elasticsearch==1.7.0
GET /haystack/modelresult/_search?_source=true [status:400 request:0.001s]
Failed to query Elasticsearch using '(code)': TransportError(400, 'parsing_exception')
Traceback (most recent call last):
File "c:\python34\lib\site-packages\haystack\backends\elasticsearch_backend.py", line 524, in search
_source=True)
File "c:\python34\lib\site-packages\elasticsearch\client\utils.py", line 69, in _wrapped
return func(*args, params=params, **kwargs)
File "c:\python34\lib\site-packages\elasticsearch\client\__init__.py", line 527, in search
doc_type, '_search'), params=params, body=body)
File "c:\python34\lib\site-packages\elasticsearch\transport.py", line 307, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "c:\python34\lib\site-packages\elasticsearch\connection\http_urllib3.py", line 93, in perform_request
self._raise_error(response.status, raw_data)
File "c:\python34\lib\site-packages\elasticsearch\connection\base.py", line 105, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.RequestError: TransportError(400, 'parsing_exception')
[28/Dec/2016 17:58:50]"GET /search/?q=code HTTP/1.1" 200 395
no [query] registered for [filtered]
From what I can see you are running ES 5.0 and you're sending a filtered query which has been deprecated in ES 2.x and removed in ES 5.x.
You need to replace it with a bool/filter query instead.
So if you had something like this:
{
"query": {
"filtered": {
"filter": {}
}
}
}
Simply replace it with
{
"query": {
"bool": {
"filter": {}
}
}
}
Elasticsearch already deprecate filtered query. Use bool instead.
It works for me.

Resources