So this was a fun problem, ElasticSearch on AWS in it’s current form is completely public and not secured within my VPC. It appears that first you have to put in a deny statement to lock off access and then you add in the roles / users that have access. But once you do this you now have a box that can’t be accessed by CURL… very annoying. So now you need to write a python (or language of your choice) script to send a signed request. Here’s the code I’m working on:

#!/usr/local/bin/python
import requests
import json
from requests_aws_sign import AWSV4Sign
from boto3 import session
from elasticsearch import Elasticsearch, RequestsHttpConnection

import sys, getopt

host = ''
json_map = ''
index = ''
try:
 opts, args = getopt.getopt(sys.argv[1:],"h",["host=","json_map=","index="])
except getopt.GetoptError:
 print '--host <ES HOST> --index <index name> --json_map <ES Mapping>'
 sys.exit(2)
for opt, arg in opts:
 if opt == '-h':
 print '--host <ES HOST> --index <index name> --json_map <ES Mapping>'
 sys.exit()
 elif opt == "--host":
 es_host = arg
 print es_host
 elif opt == "--json_map":
 json_map = arg
 print json_map
 elif opt == "--index":
 index_name = arg
 print index_name



with open(json_map) as data_file:
 request_body = json.load(data_file)

# Establish credentials
session = session.Session()
credentials = session.get_credentials()
region = session.region_name or 'us-west-2'

# Elasticsearch settings
service = 'es'
auth = AWSV4Sign(credentials, region, service)
es_client = Elasticsearch(host=es_host,
 port=443,
 connection_class=RequestsHttpConnection,
 http_auth=auth,
 use_ssl=True,
 verify_ssl=True)

# print es_client.info()
try:
 res = es_client.indices.create(index=index_name,body=request_body)
 print(" response: '%s'" % (res))
except Exception as e:
 print("Not working...")
 print(e)

My developers can now commit their JSON to git to create their index mappings and it can be auto created by Jenkins.

=== UPDATE ===
Doesn’t really work, it appeared that I had a solution but finding that it’s 90% there.. Leaving this post here for now in hopes that it might be useful at some point.

Written by kevin

1 Comment

Comments are closed.