My journey: Dockerized Java on Apache Mesos Container cluster on Azure – Setup Mesus cluster on Azure

September 2, 2016

no comments

At the end of the last post (see here) we finished when we have an “Automated Build Repository” on Docker-Hub. Now we need a containers platform to run it. I decided to use the Apache Mesus open source solution, when the cluster located on Azure cloud.

Step 1: create the Apache Mesus cluster.

  • Login to Azure Portal using this link.

azure_container_service_1

  • Press on the + New link
  • On the search box write “Azure Container Service

azure_container_service_2

  • The filter slide will open with the “Azure Container Service” text.
  • Select the DC/OS on Azure container.

DCOS_create

  • Press the “Create” button.

Step 1-Tag: create the SSH – Keys.

I put it as a Tag because it’s a sub-routing we need to preform before the DC/OS on Azure container creation.

For that I’m refereeing you to a helpful post I saw “Create SSH keys on Linux and Mac for Linux VMs in Azure“. Please navigate to the “Create the SSH Keys” section.

I summarize it with a terminal snippit.

admin-ajax

The code for it will be:

## Run the ssh-keygen tool
ssh-keygen -t rsa -b 2048 -C [User@Domain]

## Give a name to create the file - I set it to
## my_id_rsa_file

Now two files will create:

  1. my_id_rsa_file: with the private key data
  2. my_id_rsa_fiule.pub: with the public key data – This key will use us to create the DC/OS container.

⇒⇒⇒⇒⇒ Back to the  DC/OS on Azure container creation.

CreateDCOSonAzure

  • First see that only the Basic stage is mandatory.
  • Set the number of Agents to 3 or more.
  • Use that SSH public key that you created on the previous step.
  • Fill in the master, agent and resource group names.

Click Ok and create the container – wait for several minutes for the process to complete.

When deployment finished, navigate to resource-group, in this case dcos-amours-rg.

dcos-amours-rg

 

Step 2: security issues – open ports configurations.

The deployed cluster is sealed and cannot be use on this configuration. We need to add inbound ports to the configurations.

Step 2.a: Master Load Balancer configurations.

  • Click on the master Load Balancer line – this case dcos-master-lb-01234567.
  • Press on the Inbound NAT rules link.
  • Add new Rule:
    • Name: HTTP.
    • Service : HTTP (from the drop-down list).
    • Protocol: TCP.
    • Port: 80.
    • Target: Chose the Doc-Master. Here dcos-master-01234567-0.
    • Port mapping: Default.

Press OK to add it.

*** Wait to the processes to finish before continue on!!!

Add inbound NAT rule Master Process

 

Step 2.b: Master Network Security Group configurations.

  • Click on the master Network Security Group line – this case dcos-agent-public-nsg-01234567.
  • Press on the Inbound security rules link.
  • Add new Rule:
    • Name: HTTP.
    • Service : HTTP (from the drop-down list).
    • Protocol: TCP.
    • Port range: 80.
    • Action: Allow.

Inbound security rules

Press OK to add it.

*** Wait to the processes to finish before continue on!!!

Now we good to go.

Step 3: Open the DC/OS Portal.

  • Click on the master Load Balancer line – this case dcos-master-lb-01234567.
  • Press the Overview.
  • Copy the Public IP address.

dcos-master-lb-01234567IP

  • Navigate to this address

dcosportal

Step 4: Add Chronos package.

First as you saw from the Dashboard the Marathon (A container orchestration platform for Mesos and DCOS) already installed. In fact the other packages runs via the Marathon scheduler which rise it and keep it up.

On the other hand we have the Chronos, a fault tolerant job scheduler for Mesos which handles dependencies and ISO8601 based schedules.

*** If your Docker application should run all the time – always alive. The build-in Marathon is all you need. If we talk about schedule small jobs install the Chronos for it.

Install the Chronos package.

  • On the DC/OS portal press on the Universe link.
  • Install Chronos package.

InstallChronos

  • Wait for a minute or two and press the Services link.
  • You will see that both Marathon and Chronos services are running and Healthy.

MarathonnChronosRunning

Step 5: Setup Chronos job.

  • Press on the Chronos service from the Services panel.
  • Than press the “Open Service” button (purple).

OpenChronos

  • On the opened window press the + New Job button and a Card to fill the the job details will be open.

NewJob

Populate the Job Card.

  • Name: Give your job a name (Run Test).
  • Description: … (Run Demo Java Application).
  • Command: full command with the docker name as specified on the Docker-Hub. Also you can pass here the incoming parameters. I’m passing : sudo docker run amourshmuel/docker_java_app Docker_on_Mesos, when:
    • amourshmuel/docker_java_app: The Docker name on Docker-Hub.
    • Docker_on_Mesos: The incoming parameter to the Java Application.
  • Other parameters set as you go, I set the Docker to run each 5 minute (T5M).

*** Before saving it open the browser Developer Tools (F12 on Chrome). We want to capture the REST call which create the Job.

  • Press the Create button.
  • The Job will be execute and run (the iteration runs count the success and failures)

ChronosSavedAndRun

*** On the browser Developer Tools (F12 on Chrome). We captured the next REST call which create the Job.

iso8601

Step 6: Validate the Docker execution.

  • Navigate back to the Mesos portal to the Services panel.
  • Once again press the Chronos service link.

This time you will see all the executed tasks. With the task name, state,CPU, Memory and the elapsed time from the run.

ChronosTasksOut

Press on one of the tasks and you will get:

  • Data about the Docker
  • 3 tabs with the run details
    • Files: raw logs file (strand, error …)
    • Details: about the running Docker and the node which run this Docker (we used 3 node cluster)
    •  Log Viewer: the raw logs in two modes Error and  output.

Error Log will include the download of the Docker image (it’s an error that it’s not located locally). 

ChronosTasksErrorLog

Output will include the running command and the Standard output.

 ChronosTasksOutput

We can see that the executed command was: ‘sudo docker run amourshmuel/docker_java_app Docker_on_Mesos’.

The result of this run was: Hello World from: Docker_on_Mesos. (as expected).

the status is 0 and we also get the pid which execute it.

 

Summarize it.

We have an Automated Docker Build which contains Java application that can receive parameters. This Docker located on the Docker-Hub and pulled by the Mesos container nodes. The Mesos located on Azure Cloud and the we declared the jobs on Chronos which rise the nodes and DO THE JOB 🙂

Hope those posts assist you.

Add comment
facebook linkedin twitter email

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

*