Java Spark ElasticSearch driver loading invalid records
up vote
0
down vote
favorite
I'm using ElasticSearch 6.x version with Spark ES driver.
While creating the index/type, I'm using dynamic = strict mode.
When I try to load a document which contains fields other than what is mentioned in the mapping file using a rest-client (Postman), I'm getting an error as below:
{
"error": {
"root_cause": [
{
"type": "strict_dynamic_mapping_exception",
"reason": "mapping set to strict, dynamic introduction of [1URBANIZEDAREA] within [tc302_record] is not allowed"
}
],
"type": "strict_dynamic_mapping_exception",
"reason": "mapping set to strict, dynamic introduction of [1URBANIZEDAREA] within [tc302_record] is not allowed"
},
"status": 400
}
However, when I try to load the same document using Java spark ES driver, the document is going through. But it loading only those fields that are mentioned in the mapping file. Other fields are ignored. It is behaving as if dynamic is set to false.
Any ideas on how to resolve this issue?
java apache-spark
elasticsearch asked Nov 13 at 11:45
Sunil Kumar B M
2,0941324
add a comment |
up vote
0
down vote
favorite
I'm using ElasticSearch 6.x version with Spark ES driver.
While creating the index/type, I'm using dynamic = strict mode.
When I try to load a document which contains fields other than what is mentioned in the mapping file using a rest-client (Postman), I'm getting an error as below:
{
"error": {
"root_cause": [
{
"type": "strict_dynamic_mapping_exception",
"reason": "mapping set to strict, dynamic introduction of [1URBANIZEDAREA] within [tc302_record] is not allowed"
}
],
"type": "strict_dynamic_mapping_exception",
"reason": "mapping set to strict, dynamic introduction of [1URBANIZEDAREA] within [tc302_record] is not allowed"
},
"status": 400
}
However, when I try to load the same document using Java spark ES driver, the document is going through. But it loading only those fields that are mentioned in the mapping file. Other fields are ignored. It is behaving as if dynamic is set to false.
Any ideas on how to resolve this issue?
java apache-spark
elasticsearch asked Nov 13 at 11:45
Sunil Kumar B M
2,0941324
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I'm using ElasticSearch 6.x version with Spark ES driver.
While creating the index/type, I'm using dynamic = strict mode.
When I try to load a document which contains fields other than what is mentioned in the mapping file using a rest-client (Postman), I'm getting an error as below:
{
"error": {
"root_cause": [
{
"type": "strict_dynamic_mapping_exception",
"reason": "mapping set to strict, dynamic introduction of [1URBANIZEDAREA] within [tc302_record] is not allowed"
}
],
"type": "strict_dynamic_mapping_exception",
"reason": "mapping set to strict, dynamic introduction of [1URBANIZEDAREA] within [tc302_record] is not allowed"
},
"status": 400
}
However, when I try to load the same document using Java spark ES driver, the document is going through. But it loading only those fields that are mentioned in the mapping file. Other fields are ignored. It is behaving as if dynamic is set to false.
Any ideas on how to resolve this issue?
java apache-spark
elasticsearch asked Nov 13 at 11:45
Sunil Kumar B M
2,0941324
I'm using ElasticSearch 6.x version with Spark ES driver.
While creating the index/type, I'm using dynamic = strict mode.
When I try to load a document which contains fields other than what is mentioned in the mapping file using a rest-client (Postman), I'm getting an error as below:
{
"error": {
"root_cause": [
{
"type": "strict_dynamic_mapping_exception",
"reason": "mapping set to strict, dynamic introduction of [1URBANIZEDAREA] within [tc302_record] is not allowed"
}
],
"type": "strict_dynamic_mapping_exception",
"reason": "mapping set to strict, dynamic introduction of [1URBANIZEDAREA] within [tc302_record] is not allowed"
},
"status": 400
}
However, when I try to load the same document using Java spark ES driver, the document is going through. But it loading only those fields that are mentioned in the mapping file. Other fields are ignored. It is behaving as if dynamic is set to false.
Any ideas on how to resolve this issue?
java apache-spark
elasticsearch java apache-spark
elasticsearch asked Nov 13 at 11:45
Sunil Kumar B M
2,0941324
asked Nov 13 at 11:45
Sunil Kumar B M
2,0941324
asked Nov 13 at 11:45
Sunil Kumar B M
2,0941324
asked Nov 13 at 11:45
Sunil Kumar B M
2,0941324
asked Nov 13 at 11:45
Sunil Kumar B M
2,0941324
2,0941324
add a comment |
add a comment |
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53280359%2fjava-spark-elasticsearch-driver-loading-invalid-records%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
