Automating Digicert Certificates Into AWS ACM

Like most security professionals I am spending a large amount of time helping my company move securely to AWS.
Certificate management in AWS is done with AWS Certificate Manager  and while they do offer *free* certificates, ACM generated certs are outside your direct control. You don’t get the keys which, at least for some things, should probably be a non-starter (granted, for plenty of other things it’s likely  ¯\_(ツ)_/¯).
I also really like digicert and have been using them for TLS certificates for over 10 years but I could not find any automation already built for Digicert to AWS ACM so I spent some time this week and hacked a script together to do it.
Here is a link to the script  (also embedded at the bottom of the post). On the host running the script you will need AWS CLI  configured and a Digicert API Key.  You also need to configure the first 15 lines of the script with your information.

To Run The Script:

./awasacm.sh your.fdqn.com

Script Output:

Here is what the script looks like running:

Here is the cert uploaded to ACM:

The script also saves all of the commands, keys and certs on the host running the script for auditing and backup:

Full Script:

https://gist.github.com/jgamblin/f8bd03d3743ba4f08f710d5e11c177c7

Closing:

I will be making improvements to this script as we implement it in production and will likely move it to a full GitHub repo soon.   If you have any questions please reach out to me on twitter at @JGamblin. 
Update:  I have built a full Github repo here.

My Security Summer Camp Talk List

Security summer camp is about a week away so I spent some time this afternoon trying to figure out what talks and events I want to make sure I attend.
BSides Las Vegas:
A Day in the Life of a Product Security Incident Response Manager
From SOC to CSIRT
Hadoop Safari : Hunting For Vulnerabilities
Introduction to Reversing and Pwning
YARA-as-a-Service (YaaS): Real-Time Serverless Malware Detection
Abusing Webhooks for Command and Control
BSides Las Vegas Full Schedule

Blackhat:

Breaking Electronic Door Locks Like You’re On CSI: Cyber
Free-Fall: Hacking Tesla From Wireless To Can Bus
Blackhat Full Sechedule

Defcon 25:

Meet the Feds (who care about security research)
There’s no place like 127.0.0.1 – Achieving reliable DNS rebinding in modern browsers
Wiping Out CSRF
Real-time RFID Cloning in the Field
Exploiting 0ld Mag-stripe information with New technology
Secret Tools: Learning About Government Surveillance Software You Can’t Ever See
Next-Generation Tor Onion Services
Using GPS Spoofing to Control Time
Cisco Catalyst Exploitation
Defcon Full Schedule

Other Events:

IOACTIVE IOASIS
ShabbatCon
Defcon Parties List
 

Quickly Building A Cloud Virtual Lab

Often while doing research I need temporary access to a bunch of different virtual machines. While it is possible to do this on my Macbook using VMWare Fusion or Virtualbox the overhead seems unnecessary for something I will delete in under a week.
My goto solution is a virtualization stack of:
16GB DigitalOcean Droplet + Wok + Kimchi
Here is the shell script I use to build it:

#!/bin/bash
apt-get update &&  apt-get upgrade -y
apt-get -y install qemu qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils nginx python-cherrypy3 python-jsonschema python-m2crypto nginx python-ldap python-psutil fonts-font-awesome texlive-fonts-extra python-configobj python-parted sosreport python-imaging websockify novnc nfs-common python-ethtool open-iscsi python-guestfs libguestfs-tools spice-html5 python-paramiko
wget http://kimchi-project.github.io/kimchi/downloads/latest/kimchi.noarch.deb
wget http://kimchi-project.github.io/wok/downloads/latest/wok.noarch.deb
wget http://kimchi-project.github.io/gingerbase/downloads/latest/ginger-base.noarch.deb
dpkg -i wok.noarch.deb
apt-get install -f -y
dpkg -i ginger-base.noarch.deb
apt-get install -f -y
dpkg -i kimchi.noarch.deb
apt-get install -f -y
reboot
#You will need to know the root password for the web interface (passwd lets you reset it).

After the server is rebooted you can access the web interface at https://ip:8001:

The next step is to add the templates you want to build VMs for:

You can use these commands to grab newer isos (there is a feature request to automate this):

cd /var/lib/kimchi/isos
wget -c http://cdimage.kali.org/kali-2017.1/kali-linux-2017.1-amd64.iso
wget -c http://releases.ubuntu.com/17.04/ubuntu-17.04-desktop-amd64.iso
wget -c http://releases.ubuntu.com/17.04/ubuntu-17.04-server-amd64.iso
wget -c http://releases.ubuntu.com/16.04/ubuntu-16.04.2-desktop-amd64.iso
wget -c http://releases.ubuntu.com/16.04/ubuntu-16.04.2-server-amd64.iso
wget -c ftp://opensuse.mirrors.ovh.net/opensuse/distribution/13.2/iso/openSUSE-13.2-DVD-x86_64.iso
wget -c http://slackware.mirrors.ovh.net/ftp.slackware.com/slackware64-14.2-iso/slackware64-14.2-install-dvd.iso
wget -c http://archlinux.mirrors.ovh.net/archlinux/iso/2016.09.03/archlinux-2016.09.03-dual.iso
wget -c https://download.fedoraproject.org/pub/fedora/linux/releases/25/Workstation/x86_64/iso/Fedora-Workstation-Live-x86_64-25-1.3.iso
wget -c https://az792536.vo.msecnd.net/vms/VMBuild_20150801/VirtualBox/MSEdge/Windows/Microsoft%20Edge.Win10.For.Windows.VirtualBox.zip

Once you are done with that is is amazingly easy to spin up VMs and manage them in the browser:

I use this virtualization stack a lot in my research and it is amazing.  If you have any questions feel free to reach out to me on twitter.

Big Data’ing The Umbrella DNS Popularity List

Recently I started looking at the Umbrella DNS Popularity List and did a blog post about it here. The data seemed valuable and lacking at the same time so I spent my *limited* free time this week learning about R and RStudio.
Protip:  If you want to play along at home there is an RStudio docker container so all you need to do is:

docker run -d -p 8787:8787 -e USER=<username> -e PASSWORD=<password> rocker/rstudio

Getting today’s list loaded into R is as simple as:

# Get Todays List
if (file.exists(fn)) file.remove(fn)
temp <- tempfile()
download.file("http://s3-us-west-1.amazonaws.com/umbrella-static/top-1m.csv.zip",temp)
unzip(temp, "top-1m.csv")
today <- read_csv("top-1m.csv", col_names = FALSE)
unlink(temp)

Now you have the Top 1 million DNS requests from Umbrella ready to be “big data’ed”.
At the start of this project I wanted to do the following:
Search the DNS names for keywords. (Done).
Map all the DNS records on a map. (Done, Kinda).
Compare today’s and yesterday’s records for new DNS records.
Check all the DNS records against Censys and record open ports, and software.
Check all the DNS records against VirusTotal and see if any of them are known bad.
Check all the DNS records against SSLLabs and record SSL grade.
Take a nap.
My limited results so far follow with hopefully more to come.

Search The DNS Names

I wanted to do this to be able to search the list for a keyword and build a table and map of the data.  This was fairly easy and with help of leaflet and datatables here is the output of searching today’s data for cisco.
Here is the map:

Here is a link to the data. 
Here is the R code I wrote:
https://gist.github.com/jgamblin/7615b81cedd10e44d4f2220347b69cb0

Map All The DNS Records On A Map.

I got started on this and quickly realized that looking up the GEOIP information and mapping a million DNS records was going to take a week so I decided to do the Top 25,000 as a POC and come back and do all 1,000,000 later (maybe).
Here is the 25,000 Map:
Here is the R code I wrote:
https://gist.github.com/jgamblin/ccf3390bc5d2ce922cd5df38a40617b4
I also built a map with the Top 100K on it but it is huge (Load at your own risk).

…More to come.

I will be spending some more time on this over the next couple of weeks but cant think @EngelhardtCR and @hrbrmstr enough for all the help they have been over the last week as.   They are true data scientist and I am just a hacker with a blog.  : )
If you have any questions or suggestions please let me know on twitter at @jgamblin.
Here is a picture semi related to this blog post to make it look pretty when I share it on social media. 

Exploring Cisco’s Top 1 Million Domains Data

Cisco offers a daily list of the million most queried domain names from Umbrella (OpenDNS) users.    I had some time this weekend so decided to spend some time playing around with the data to see what I could find so I spun up a lightsail server and got to work.
Grabbing the file is as simple as:
wget http://s3-us-west-1.amazonaws.com/umbrella-static/top-1m.csv.zip
You can retrieve a specific date like this:
wget http://s3-us-west-1.amazonaws.com/umbrella-static/top-1m-yyyy-mm-dd.csv.zip
(Looks like 2017-01-20 is the earliest they have online).
Once you get that downloaded and unzipped (unzip top-1m.csv.zip) you can start exploring.
You can pull out the top 10 domains with this command:
head -n 10 top-1m.csv

1,google.com
2,www.google.com
3,microsoft.com
4,facebook.com
5,doubleclick.net
6,g.doubleclick.net
7,clients4.google.com
8,googleads.g.doubleclick.net
9,apple.com
10,fbcdn.net

(Full Output)

You can search for keywords with this command:
cat top-1m.csv | grep "opendns"

437,opendns.com
719,hydra.opendns.com
720,sync.hydra.opendns.com
1314,disthost.opendns.com
2756,api.opendns.com
4565,cacerts.opendns.com
5569,ipf.opendns.com
5699,block.opendns.com
7024,updates.opendns.com
8482,bpb.opendns.com

(Full Output)

To count the domain levels use this command:
awk -F, '{count=split($2,a,"."); print count}' top-1m.csv | sort | uniq -c | awk '{print $2,$1}' | sort -k1,1n

1 1086
2 263509
3 469756
4 193802
5 54281
6 13698
7 2952
8 689
9 172
10 16
11 26
12 2
13 1
14 1
15 1
16 1
17 1
18 1
19 1
20 1
21 1
22 1
23 1

(Full Output)
Notice anything strange here? Hint: A domain name requires at least two levels to be valid.

To find the broken DNS names in this list this command works:
cat top-1m.csv | awk -F, 'BEGIN {file="top-1m.csv" ; while ((getline line < file) > 0) {if (line ~ /#/) continue; tld[tolower(line)] = 1}} {foo=split($2,a,"."); if (foo == 1) {if (!(a[1] in tld)) {print $0}}}'  

1200,home
1490,local
2082,za
3916,lan
6350,url
10173,belkin
10869,uop
11187,localdomain
12887,localhost

(Full Output)

Find domains added to the list for today.
I  wrote a script to download the last two days of files and compare them for new domains:
https://gist.github.com/jgamblin/184590e2ba64371730e435ab2977e4cf

You can find the output for April 24, 2017 here.

Overall I am really impressed with this data and will be using it to do more research and to track trends across the internet.  They have some more to do but it is an amazingly valuable free tool.
Also recently I have feel in love with sprunge to push data to an ad free “pastebin” from the command line:

cat file.txt | curl -F 'sprunge=<-' http://sprunge.us

Automating DigiCert Certificate Issuance

I am a big fan of DigiCert for TLS Certificates and CA/WebPKI services.   While they have amazing customer support and are an amazing company to work with, there are not a lot of automation scripts to interact with their API available. So over the weekend and with a lot of help from Clint Wilson I built a shell script that:

  • Creates a CSR/Key pair using OpenSSL.
  • Uses the Digicert API to:
    • Request a TLS certificate.
    • Approve the certificate.
    • Download the certificate in:
      • .zip
      • p7b
      • pem
      • pem (with no root)

Here is the script in action:

Here is the code:
https://gist.github.com/jgamblin/bd04b9ef8fe3660f4a247cc7d2109df0
I have tested it on OSX,  Ubuntu and CentOS7 and it is fairly cross platform friendly.  Extending this script to install it should be easy but we already had the automation built to do that so it was not necessary.
Let me know on twitter if you have questions.

Leadership Quotes From My Mentors Dad

An amazing mentor and leader I work with has been talking to me recently about what real leadership looks like and shared with me a list of quotes he keeps on his desk that his dad who had a leadership role in the military collected and gave to him.  He gave me a copy and said I was free to share them.

My [Dad’s] Rules Of Leadership:
  • Develop a vision and live it.
  • Dont lie for your people and dont lie to your people.
  • Beware of RUMINT.  It’s faster than you are.
  • Dont back away from the hard decisions,  especially personnel decisions.
  • Bad news never goes down easy and it won’t get easier with time.  It’s best to get it over with.
  • Support your subordinate supervisors when they take the high road.
  • Tell people exactly what you expect of them, including the obvious.
  • Involve your people in decisions and action planning.
  • Give them credit when things work.  Give them top cover when things go awry.
  • Trust the experts.  That’s what you pay for them for.
  • Avoid Bullshit.  You may get past the fans, but you won’t get past the players.
  • Knowledge may be power, but knowledge shared is powered squared.
  • Set an example by taking on the hard jobs.
  • Listen-Decide-Explain-Act.
  • Old ship-driving rule: When you get in extremis, DO SOMETHING.  The worse thing you can do is nothing.  Make decisions smartly and dont vacillate.  If you are wrong, admit it, back up and turn right.
  • No, you are not always right.  Get over it.  You are not as smart as you think you are and you may not be as smart as others think you are either.
  • Never underestimate the power of the expression “Thank You”.
  • Don’t fight with your friends. You haven’t got the time.

These quotes are amazing and I will be reflecting on this list for the rest of my career and am really happy to be able to share them.
Here is a “leadership” picture so shared links look better:

A Threat Intelligence Thought Exercise

I was at dinner on Tuesday with 6 security professionals and I proposed this hypothetical situation and I thought it was worth writing up and sharing.

Background:

  • Six identical safes with $1,000,000 inside are being built into the side of a public building and are being randomly assigned to everyone at the dinner.
  • At the end of 90 days any money left in your safe is yours.
  • You will be given a live video feed of your safe.
  • There is an advanced and persistent team of safe crackers trying to crack all six safes.
  • You are loaned $100,000 to spend on security for your safe that must be repaid when  the project is over.
  • Everyone at the dinner is your friend.

Threat Intelligence Questions:

  • While doing video monitoring you notice that you see activity that you think is coming from the safecrackers every Monday night from 2100 to 2200.
    • Do you tell the other safe owners?
  • While looking at your safe you try the passcode 8675309 because the song is stuck in your head and find it is an unknown one time backdoor.
    • Do you tell the other  safe owners?
  • You are approached by 2 other safe owners who would like to form an alliance with where you will share all information you have on threats and at the end of the 90 days you split any remaining money evenly.
    • Do you join the alliance?
  • Does sharing what you know with the other safe owners make you more or less secure? 
  • Does joining the alliance  make you more or less secure? 
  • If 4 people are in the  alliance does that make you more or less secure? 

Bonus Traditional Security Questions:

  • A physical security firm will place an unarmed but well meaning guard in front of your safe from 0700 to 1900 every day for the 90 days. The cost is $40,000.
  • A monitoring firm will monitor the video feed of your safe for the 90 days and send you a text message if something looks wrong. The cost is $40,000.
  • A famous ex-hacker will spend two weeks trying to break into your safe and another week writing a report you will only kind of read. The cost is $60,000.
  • A company has a team of 100 less skilled but trust worthy safecrackers who they can have try to break into your safe. They will run the program for you, pay a reward to anyone who successfully cracks your safe and tell you exactly how the did it. The cost is $60,000.
    • How do you spend your money?
  • On day 30 you are told only one safe has been successfully broken into.   You have not spent any of your money.
    • Do you buy anything now? 
  • On day 60 you are told only one safe has been successfully broken into.   You have not spent any of your money.
    • Do you buy anything now? 

I think there are a bunch more questions I could ask but I think this is a great thought exercise to think through and discuss.
If you have any thoughts you can share them with me on twitter @jgamblin. I just do not have the patience to moderate blog comments.
Here is a picture of a safe so my links look better when I share this:

Digital Ocean Slack Bot

I had a coach whose favorite quote was “Pain is the best teacher.”  and that was the first thing that popped into my head this morning when I realized that I had left an $80 a month  Digital Ocean Droplet  running for an extra 3 weeks after I got done using it.   To be honest $60 isn’t *that* painful but  it did prod me to write a slackbot that will post all my running droplets to slack once a day:
 

Here is how I set it up:

https://gist.github.com/jgamblin/83f7eb018d7953497e30802ed23e2041

  • Copy this line to your crontab to post a list of running droplets at 0800 everyday:
    00 08 * * * /path/to/dobot.sh
  • Enjoy not wasting money on abandoned cloud servers.

8 Security Predictions for 2017

 

What will 2017 hold for the security industry?    I sat down and looked into my crystal ball and came up with these 8 security predictions for 2017. 

A Fortune 500 Will Use “DDOS as a Service” To Attack A Competitor.
A bored VP of Marketing with a paypal account, a six pack and a nephew who can get him on the “undernet” is the cyber warrior of the future.
Internet of Thing (IoT) will continue to be used as an attack platform.
Who would have thought that un-patched, un-maintained linux operating systems exposed to the internet would be used to do evil?
Hacking As A Service Will Take Off.
Want into your bosses/spouses/political opponents email account? A hacking group will sell you access for $500.
Hackers Discover & Exploit Automation Platforms.
Hackers finally realize they only have to own the automation platform (Chef, Puppet and SaltStack) of most companies to own the whole company.
Red-Teaming Will Still Be easy.
…and there will be 437 conference talks about how awesome it is.
A Killer Android Bug Will Be Found.
A remotely exploitable android bug that allows for remote camera, microphone and speaker access will be found and will be un-patchable on 75% of phones. Causing Google to take control of the OS and push patches to all phones.
Hackers Will Interrupt A Major Sporting Event.
Just think what would happen if the Russians, Chinese or a 400-pound hacker sitting on his bed decided to take Fox offline during the opening kickoff.
Security Will Still Be Hard.
…and no one as invented the magic box you can plug into your network and make it easy.
Remember though… you can change the future of security in 2017 by working hard and doing something that makes a difference.  Get involved in the EFF,  write some amazing open source software,  volunteer to mentor someone interested in security.

Site Footer