Monday, September 24, 2018

Data Science environment with Docker and Jupyter on the IBM Mainframe

Guide to getting started with Docker, Python and Jupyter Notebook on zLinux.

Here, I'm using Red Hat Enterprise Linux 7.5 to build and deploy Jupyter notebook in an Ubuntu container. I will go over the steps used to build and run a Docker container.
Docker Containers on the IBM Mainframe - LinuxONE
Oh, and in case you're wondering: why would anyone do this - check out this snippet from the z14 announcement: "Microservices can be built on z14 with Node.js, Java, Go, Swift, Python, Scala, Groovy, Kotlin, Ruby, COBOL, PL/I, and more. They can be deployed in Docker containers where a single z14 can scale out to 2 million Docker containers".

A few basic commands:

Establish the OS release and version. We're running on RHEL 7.5 for s390x.
[cmihai@rh74s390x ~]$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.5 (Maipo)

[cmihai@rh74s390x ~]$ uname -a
Linux rh74s390x.novalocal 3.10.0-693.17.1.el7.s390x #1 SMP Sun Jan 14 10:38:29 EST 2018 s390x s390x s390x GNU/Linux

[cmihai@rh74s390x ~]$ docker --version
Docker version 17.05.0-ce, build 89658be

Setup regular user access, sudo and SSH keys

Create a regular user account

useradd cmihai
passwd cmihai
usermod -aG wheel cmihai
su - cmihai

Add your SSH public key to authorized_hosts

mkdir -p ~/.ssh
echo "YOURKEYHERE" >> ~/.ssh/authorized_keys

Log in as your new user, and forward port 9000:

ssh -L 9000:https://www.linkedin.com/redir/invalid-link-page?url=127%2e0%2e0%2e1%3A9000 -i cmihai.pem cmihai@myzLinux

Setup docker

Create the Docker group

sudo groupadd docker
sudo usermod -aG docker cmihai

Start Docker

sudo systemctl enable docker
sudo systemctl restart docker.service
sudo systemctl status docker.service

Test docker

docker run s390x/hello-world

Let’s run a simple Ubuntu interactive shell:

docker run --name s390x-ubuntu --hostname s390x-ubuntu --interactive --tty s390x/ubuntu /bin/bash

Building a Docker container for Jupyter Notebook

Create a Dockerfile from the s390x/ubuntu base image.
FROM s390x/ubuntu
MAINTAINER Mihai Criveti

# ADD AND RUN
RUN apt-get update \
    && apt-get install -y python3 python3-pip \
    && pip3 install jupyter \
    && apt-get clean

# COMMAND and ENTRYPOINT:
CMD ["jupyter","notebook","--allow-root","--ip=0.0.0.0","--port=9000"]

# NETWORK
EXPOSE 9000

Build the container:

docker build . --tag "cmihai/jupyter-lite:v1" -f Dockerfile

Run your new container:

docker run --name jupyter --hostname jupyter -p 9000:9000 cmihai/jupyter-lite:v1

Connect to Jupyter Notebook

You can now install depedencies directly from Jupyter:

!apt-get install --yes zlib1g-dev libjpeg-dev
Potential next steps:
  • Consider setting up persistence for your notebooks (ex: VOLUME ["/notebooks"] in Dockerfile)
  • Setup Docker Compose and build multi-tiered applications specifications - such as connecting your Jupyter Notebook to PostgreSQL, Redis, Spark, etc.
  • Set up other programming languages or kernels (Java, R) even Zeppelin Notebook
For an interactive tutorial of using Docker for Data Science, check out: https://github.com/crivetimihai/docker-data-science

Monday, August 13, 2018

Learn Cloud Native, Docker, K8s & Istio with free courses and IBM Badges

Microservices are the building blocks for cloud native architecture. Learn how to create and use containers with Docker, manage and orchestrate Containers with Kubernetes and secure and connect microservices with Istio.

Prerequisites:

You should have a good understanding of Linux and some basic concepts such as Version Control (preferably using Git). If you need a refresher, check out these 2 free resources:
  1. Optional: Learn Git Version Control on Katakoda if you've never used Version Control.
  2. Optional: Take up Introduction to Linux by The Linux Foundation on edX.

Get started learning containers with Docker

Find out what containers are, how they differ from Virtual Machines and what are the benefits of using containers. Install Docker CE on your machine, search for and run container from the Docker Hub, build your first Docker container from a Dockerfile and publish it to a Docker Registry.
  1. Learn Docker & Containers using Interactive Browser-Based Scenarios on Katakoda.
  2. Test your knowledge and collect the Docker essentials: Extend your apps with containers badge on developerWorks.
  3. Optional: Try my interactive Docker for Data Science course.

Learn container orchestration with Kubernetes

Kubernetes is a platform for managing containerized workloads and services. Learn about the key Kubernetes components and architecture:
  1. Join Introduction to Kubernetes by The Linux Foundation on edX.
  2. Collect the Container & Kubernetes Essentials with IBM Cloud badge.
  3. Complete Kubernetes Introduction on Katakoda.

Cloud Native, Microservices, 12-factor and Istio

Learn about essential cloud-native technologies, the twelve-factor app methodology, microservices, and Istio : an intelligent service mesh for microservices. Istio helps you to connect, secure, control, and observe services.

  1. Getting started with Microservices with Istio and IBM Cloud Kubernetes Service badge.
  2. Complete Beyond the Basics: Istio and IBM Cloud Kubernetes Service badge and also collect the badge for completing the Containers, microservices, Kubernetes, and Istio on the CloudLearning Path.
  3. Complete the Istio course on Katakoda - use the Istio service mesh to connect, manage, and secure microservices.
  4. Optional: Pursue the IBM Cloud Garage Method Explorer and IBM Cloud Garage Method Advocate badges and learn about key practices such as IBM Design Thinking, Agile, DevOps used in developing and managing Cloud Native applications.

Advanced Courses & Next Steps:

  1. Complete Debugging and Troubleshooting Containers on Katakoda.
  2. Learn Docker in Production on Katakoda.
  3. Learn Docker Security on Katakoda.
  4. Learn Docker Orchestration / Swarm Mode on Katakoda.
  5. Running Containers without Docker on Kotakoda.
  6. Configuration Management for Containerized Delivery on edX.

For IBMers:

For IBM-ers who have access to YourLearning courses and Safari Books Online:
  1. Complete the IBM Cloud Private Consultant Bootcamp, which includes 15+ hours of self-paced learning on Kubernetes, Helm, Docker, Microservices (IBM Garage Method), Cloud Foundry, and introduction to IBM Cloud Private.
  2. Complete other courses and certifications from the ICp series: IBM Cloud Private - Continuous Integration/Continuous Delivery Pipelines.

Books:

  1. Read up on Docker on Safari Books online: Docker: Up & Running , Using Docker
  2. Check out: Kubernetes: Up & Running and Kubernetes Cookbook.

Next Steps:

  1. Find and join a Docker or Kubernetes community at meetup.com. Attend a few presentations, talk to people!
  2. Capstone project: find an application you like, create a Docker container for it and publish it on Docker Hub.
For additional Learning Paths, check out my Learn Cloud and AI with IBM Badges article.