Skip to main content

ElasticSearch cluster SSL/TLS configuration

ElasticSearch X-pack documentation a good description on how to secure your ElasticSearch cluster using SSL/TLS.
I used certgen to generate certificates for all the nodes as below:

  1. Create a instances.yml file:
    vim /work/elk/elasticsearch-5.6.2/config/x-pack/instances.yml

    instances:
  - name: "hostname-00"
    ip:
      - "192.126.0.163"
      - "192.0.2.2"
      - "198.51.100.1"
    dns:
      - "hostname-00"
      - "hostname-00.mydomain.name"
  - name: "hostname-01"
    ip:
      - "192.126.0.164"
    dns:
      - "hostname-01"
      - "hostname-01.mydomain.name"
  - name: "hostname-02"
  - name: "CN=hostname-03,C=GB,ST=Greater London,L=London,O=OrgName,OU=OrgUnit,DC=mydomain,DC=com"
    dns:
      - "hostname-03.mydomain.name"
      - "hostname-03.internal"
      - "hostname-03"
  2. Run below command to generate a CA certificate and private key as well as certificates and private keys for the instances that are listed in the YAML file:
    /work/elk/elasticsearch-5.6.2/bin/x-pack/certgen --days 3650 --keysize 2048 --in /work/elk/elasticsearch-5.6.2/config/x-pack/instances.yml --out /work/elk/elasticsearch-5.6.2/config/x-pack/certificate-bundle.zip
cd /work/elk/elasticsearch-5.6.2/config/x-pack/
unzip certificate-bundle.zip -d ./certificate-bundle/
    Please check certgen page for all available options.
  3. If you have already got CA certificate and key, then you can use it to sign all the generated certificates:
    /work/elk/elasticsearch-5.6.2/bin/x-pack/certgen --days 3650 --keysize 2048 --in /work/elk/elasticsearch-5.6.2/config/x-pack/instances.yml --out /work/elk/elasticsearch-5.6.2/config/x-pack/certificate-bundle.zip --cert /work/elk/elasticsearch-5.6.2/config/x-pack/certificate-bundle/ca/ca.crt --key /work/elk/elasticsearch-5.6.2/config/x-pack/certificate-bundle/ca/ca.key
    This comes in handy, when you later decide to add new nodes or clients to your cluster.
Labels:
Location: London, UK

Comments

Post a Comment

Popular posts from this blog

wget and curl behind corporate proxy throws certificate is not trusted or certificate doesn't have a known issuer

If you try to run wget or curl in Ununtu/Debian behind corporate proxy, you might receive errors like: ERROR: The certificate of 'apertium.projectjj.com' is not trusted. ERROR: The certificate of 'apertium.projectjj.com' doesn't have a known issuer. wget https://apertium.projectjj.com/apt/apertium-packaging.public.gpg ERROR: cannot verify apertium.projectjj.com's certificate, issued by 'emailAddress=proxyteam@corporate.proxy.com,CN=diassl.corporate.proxy.com,OU=Division UK,O=Group name,L=Company,ST=GB,C=UK': Unable to locally verify the issuer's authority. To connect to apertium.projectjj.com insecurely, use `--no-check-certificate'. To solution is to install your company's CA certificate in Ubuntu. In Windows, open the first part of URL in your web browser. e.g. open https://apertium.projectjj.com in web browser. If you inspect the certifcate, you will see the same CN (diassl.corporate.proxy.com), as reported by the error above ...
Post a Comment
Read more

Kafka performance tuning

Performance Tuning of Kafka is critical when your cluster grow in size. Below are few points to consider to improve Kafka performance: Consumer group ID : Never use same exact consumer group ID for dozens of machines consuming from different topics. All of those commits will end up on the same exact partition of __consumer_offsets , hence the same broker, and this might in turn cause performance problems. Choose the consumer group ID to group_id+topic_name . Skewed : A broker is skewed if its number of partitions is greater that the average of partitions per broker on the given topic. Example: 2 brokers share 4 partitions, if one of them has 3 partitions, it is skewed (3 > 2). Try to make sure that none of the brokers is skewed. Spread : Brokers spread is the percentage of brokers in the cluster that has partitions for the given topic. Example: 3 brokers share a topic that has 2 partitions, so 66% of the brokers have partitions for this topic. Try to achieve 100% broker spread...
1 comment
Read more

ElasticSearch pipeline bucket selector aggregation

ElasticSearch has a concept of bucket selection generated from aggregation. This works as a pipeline, where first aggregation generates buckets, and then bucket selection further filters out buckets. We have an ElasticSearch index ' daily_reports ', where a row represents a particular version of report. When a report is created a new row is inserted in the index with a new ' reportId ' field value and ' publishDate ' field representing the UNIX timestamp. Each report/row has multiple other fields representing properties of the report, for e.g., ' title ', ' activity ', ' reportStatus ', ' reportLevel ', etc. When the report is edited/deleted, a new row is inserted into the index, with same ' reportId ', but different '_id', 'publishDate', 'reportLevel' etc. Now if user wants to get the latest version for each report matching a particular filter criterion ( reportLevel = Monitoring AND repor...
Post a Comment
Read more