Unleash the power of the NTNX-AVM – daily_health_report and monthly_ncc_health

This series describes the installation, configuration and usage of the NTNX-AVM. In addition the mostly used scripts will be explained in detail.

full course
  1. Nutanix automation VM ( NTNX-AVM ) goes online
  2. Unleash the power of the NTNX-AVM – daily_health_report and monthly_ncc_health
  3. Unleash the power of the NTNX-AVM – Import/Export/Migrate VMs to/from AHV
  4. Unleash the power of the NTNX-AVM ‚Äď Move VM from one container to another container (AHV) with move_vm

I asked myself how could an admin use the NTNX-AVM. So I decided to show and provide some real world examples how this powerful automation VM can be used.

USE CASE: A daily health report should run on the Nutanix cluster and send to a specified email address!

Let’s start with the script itself. There is no script provided by Nutanix except the Nutanix Cluster Check (ncc). It does a decent job but because of the hundred of tests and output it may not be the easiest to start with. So based on the script provided by BMetcalf¬†in the Nutanix Community I developed a script¬†called “daily_health_report.sh” for the NTNX-AVM. It is automatically installed with the NTNX-AVM starting today.

It runs the following  command remote on a CVM which gives you a good overview of the current cluster status.

Okay we do have a script but how to run it once a day? For this case I introduced jobber to the NTNX-AVM.

Learn jobber the fast way

Connect via SSH to the NTNX-AVM and run:

jobber_list

In this case no job is known. I prepared an example which runs the script daily_health_report.sh every day at 04:00.

The easiest way to create this job is to copy the example from the source folder to a file called “.jobber” in the Nutanix home directory.

The last step is just to reload the jobs defined in “.jobber”.

Review the jobber list.

jobber_finished

How does the daily_health_check.sh work?

First of all, this script will not run in your environment because all parameters for the daily_health_check.sh are setup for my lab environment. Okay lets make sure it will run in your environment.

STEP1 – Enable SSH access from NTNX-AVM to the cluster CVMs

The script makes use of ssh/scp to run the commands remote on one of the CVMs. To run a script non-interactive we need to enable password-less authentication between the NTNX-AVM to the CVMs. I wrote a script which enables password-less authentication.

This scripts creates a key pair and deploy the keys to the CVMs. When you run it you need to specify the Cluster IP/Name and the PRISM admin password.

A test ssh connection should work now without requesting a password.

STEP2 –¬†Edit the jobber file

Use an editor of your choice like “vi” and edit the line which starts with ” cmd : ¬†daily_h…” and edit the parameters to your needs.

DO NOT use the cluster IP for host. Use one CVM IP.

  • –host=<YOUR-CVM-IP>
  • –recipient=<RECIPIENT-email>
  • –provider=other ¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬† // choose other to send email to a local email server
  • –emailuser=<EMAIL-USER> ¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬†// ¬†Email user used to authenticate¬†via SMTP (sender)
  • –emailpass=<EMAIL-PASSWORD> ¬†¬†¬†// ¬†Email password¬†used to authenticate¬†via SMTP
  • –server=<EMAIL-SERVER-IP> ¬† ¬† ¬† ¬† ¬† ¬† // Email server IP
  • –port=<SMTP port> ¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬†// Email server SMTP port

Reload the jobber file.

STEP3 – Test the job

The output should look like this:

edit_test_jobber

Check the email Account that the email was sent and received.

show_daily_email

There it is. Sorry for the German Thunderbird version but you should get the idea how the email looks like. An email with one attachment called “daily_health_report-<DATE>.txt”.

USE CASE: Run a monthly¬†“ncc health_checks run_all”¬†and send the output to a specified email address!

Some Nutanix people would say: “Why don’t you use the ncc instead?” Good point. This post shows how to run ncc every x hours and send an email. But how to run ncc once a month and get all ERROR/FAIL messages in the body?

For this case I created the ncc_health_report.sh¬†script which runs the “ncc health_checks run_all” and send an email.

STEP1 – Extend the “.jobber” file to add this job

The example¬†which can be found on the NTNX-AVM in ¬†“~/work/src/github.com/Tfindelkind/automation/NTNX-AVM/jobber/example/monthly_ncc_health”¬†defines a job which runs on the 1st of each month and calls the ncc_health_report.sh

Edit the ~/.jobber file and add the job text¬†to the end of the file. BUT skip the first line “—“. the file should look like this.

both_jobs

Don’t forget to edit the parameters like in STEP2

Reload the jobber file.

STEP3 – Test the job

WARNING!!!! This may run for a while…

The output should look like this:

ncc_report_test

And an email should be in your inbox:

ncc_report_test_email

I know the format of the body is wired because all “newlines” have been removed. I hope to fix this in the near future.

BTW: I used the hMailServer in my lab environment. This was really the easiest mail server setup I have ever done.

Go hMailServer