Darling: Continuously monitoring threats

Status: WIP

Conclusion & Implementation

Let's start with our first Module "Continuously monitoring threats". I want to figure out what it means to monitor threats and what frameworks are available to me in order to do so. After my initial research we now know that there are terms such as threat intelligence which are used to gather information about cyber threats and threat hunting which is considered to be a proactive activity.

---

System hardening

System hardening is simply configuring a system in a way to maximize the security on a server and/or a client. I think that this is a process that has to be done continuously since systems and attack methods change. For the sake of simplicity I am going to do it once. This is important in case our application has a vulnerability that could lead to an RCE.

For further investigation we can use linenum which is a great enumeration tool for Linux. It screams at us if we mess up some configurations. However, Linux is pretty secure out of the box so we are good to go as not many services have been (mis)configured.

---

Load balancing

Our business is crucial. Imagine people REALLY NEED TO KNOW WHAT TO EAT and our server is down. Well I guess there will be no food then. They will be completely lost. To keep our customers happy, I am going to configure a Docker cluster which is orchestrated by Kubernetes. Nice!

So first we pull an already existing docker image that is preconfigured with "Meinheld" and "Gunicorn" and edit our dockerfile

FROM tiangolo/meinheld-gunicorn:python3.8

COPY requirements.txt ./
RUN python3 -m pip install --upgrade pip
RUN python3 -m pip install -r requirements.txt


ENV LISTEN_PORT 8080
EXPOSE 8080 22

COPY ./app /app

Now we build the image an run it

sudo docker build --tag darling ./
sudo docker run -p 8080:8080 -e PORT="8080" darling 

Pretty neat hmm... We got our app running within seconds in an high performance environment. Unfortunately it runs on port 8080 which is an unencrypted channel. To fix this we can use a load balancer and create our SSL certificate there.

[Hello It's me from the future] The previous screenshots are now 2 weeks old and not precise. I built that whole thing with Docker and Kubernetes and got the chance of using an NGINX ingress controller. I learned so much about the Docker CLI and how to build my own Docker image from scratch and learned how to use kubectl and yaml files to deploy me services to pods. It really looked scary to me at a first glance, but was kind of easy when I got used to it. This was one hella journey but probably too big for this note. I will cover this in an other note. So just lets assume this one works like charm and everything is ready to go and protected by an SSL certificate.

UPDATE

It's quite late now but I really want to document this before I forget what I did. Unfortunately I did not manage to expose my Kubernetes Cluster through my NGINX Ingress Controller with an SSL certifiicate and I found out why. Apart from the challenge that the used processor architecture is arm64 there was a network connectivity issue. From my master node I could not reach the Cluster through the public IP, couldn't route it internally and the ACME challenge (self check part) could not be solved because of this. This is how I configured, solved and scaled my cloud infrastructure.

Implement a SIEM

Our first module task is to successfully implement a SIEM. A SIEM provides real time data for analysis of security alerts. This means that we can send all our Nginx logs to our SIEM and analyse the requests that have been sent to our webserver. This can be extremely useful if you have multiple web services and/or servers that are accessible to the world. Additionally it provides us a nice visual representation of our logs to make it easier to analyse possible threats.

When it comes to evaluate software my approach is to search for "best <software> 2020" and compare the lists I get. Then I keep track of companies that are listet several times and compare them. My top list currently is:

- Splunk
- ELK
- OSSIM

I concluded that Splunk and ELK are the best options for me. I will configure both soon and check what they are able to do.

Since our service is now up for several weeks there are already lots of logs which we can read trough. Also if we have several servers it can get inconvenient real quick. So make it easier for us to read all the logs, we can configure a SIEM.

I evaluated ELK and Splunk as the best fit for my uses sinctarpitse there are free versions and the products per se are market leaders. I'm going to install Splunk first and the ELK at a later time.

So I just downloaded Splunk from https://www.splunk.com/en_us/download/splunk-enterprise.html#tabs/linux and installed it on my local system. Unfortunately there is no free edition for ARM processors so I can't run it on my raspberry pi. But there is a universal forwarder which can run on the raspi. An universal forwarder is simply a Splunk agent that sends machine data to the server. So this is a requirement since all our services run on raspberri pi's.

So after I downloaded the tar from the Splunk website I can run:

/opt/splunk/bin/splunk start –accept-license

Next after you set your username and password, the Splunk server is available on http://127.0.0.1:8000/en-US/account/login

Last step is to configure a listener on the server to accept the data. I just followed this tutorial https://www.splunk.com/en_us/blog/iot/how-to-splunk-data-from-a-raspberry-pi-three-easy-steps.html

That setup was quick! Our server is already configured! But wait thats more. Now we want some data on in our pannel. This can be achieved with installing the Splunk universal forwarder on our machines.

Now I have learned that Splunk barely supports Raspberries. I do not have any x64 servers so currently it is not an option. But to complete this part. I will just install everything on my local machine.

So after the installation of the forwarder I need to run to receive some logs

./splunk add forward-server 192.168.0.122:9997
./splunk set deploy-poll 192.168.0.122:9997
./splunk add monitor /var/log
./splunk restart

Then the next step is to uninstall everything because nothing works correctly because of incompatibility of some components. Neat!

Let's move over to ELK and hope we get some better results there. ELK uses Elasticsearch, Logstash and Kibana to provide you log indexing, services and visualization. You need to install every component separately.

Installing Elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/current/deb.html
Installing Logstash: https://www.elastic.co/guide/en/logstash/current/installing-logstash.html
Installing Kibana: https://www.elastic.co/guide/en/kibana/current/install.html

Also not working. Nice! I think I am getting a desktop to install my SIEM. Hopefully I can get Agents working on my raspis.

Notes & Articles

https://www.networkaccess.com/cyber-threat-monitoring/
If you think your business is too small to be a target, you’re wrong. In fact, small and midsize businesses are more likely to suffer data breaches because they have their guards down and look like easy targets.
Interesting quote, but unfortunately it's not backed by anything.

Interesting! There are some definitions of threat monitoring:
Threat detection: Are there known attackers that are trying to interact with my network? (This is a weird one. I wonder what he is trying to tell me. you won't just identify an attacker as a person, are you? Maybe he is referring to network traffic or specific exploits?)
Behavioral monitoring: Monitor the users
Security Intelligence: Are there any active treats in my network (He might be talking about traceable CVE's or already ongoing threats that have been noticed)
Asset discovery: What devices are on my network?
Vulnerability assessments: What vulnerabilities are in my network? (He definitely talks about known CVE's or improperly configured software. Of course it needs to be found)

https://en.wikipedia.org/wiki/Cyber_threat_intelligence
types of threat intelligence:
Tactical: IoC (Indicator of compromise) such as IP addresses, hashes and filenames can be used as identification of threat actors
Operational: Details of the motivation or capabilities of threat actors including their tools, techniques and procedures
Strategic: Intelligence about overarching risks associated with cyber threats which can be used to drive high level strategy

https://en.wikipedia.org/wiki/Indicator_of_compromise
an IoC is an artifact on a network or device that indicates an intrusion with high confidence. Those can be virus signatures, IP addresses, hashes of malware, URLs, domain names of CNC servers

https://www.youtube.com/watch?v=qrZsc5IkchI
Yara rules: It is kinda of a fancy programming language to match certain strings and identify malware. More

https://en.wikipedia.org/wiki/Cyber_threat_hunting
https://cybersecurity.att.com/blogs/security-essentials/threat-hunting-explained
Cyber threat hunting is an activity to proactively search for new threats in the wild and/or your own network. You are not relying on security services such as firewalls, IDS, malware sandbox, and SIEMs which involve evidence-based data after an incident already happened.

Setting up Docker and Kubernetes

https://www.nakivo.com/blog/install-kubernetes-ubuntu/
https://iotbytes.wordpress.com/python-flask-web-application-on-raspberry-pi-with-nginx-and-uwsgi/
https://kubernetes.io/docs/concepts/services-networking/service/
https://networkchuck.com/kubernetes/ https://kubernetes.io/docs/tutorials/stateless-application/expose-external-ip-address/ https://www.patrickriedl.at/secure-azure-kubernetes-with-lets-encrypt-certificates/
https://kubernetes.github.io/ingress-nginx/deploy/
https://kubernetes.github.io/ingress-nginx/deploy/baremetal/
https://stackoverflow.com/questions/64125048/get-error-unknown-field-servicename-in-io-k8s-api-networking-v1-ingressbacken https://askubuntu.com/questions/1189480/raspberry-pi-4-ubuntu-19-10-cannot-enable-cgroup-memory-at-boostrap

Module tasks M681:

- Integrate a SIEM [Evaluated Splunk and ELK]
- Implement system hardening [Done]
- Implement network security: STP, VLANs, Firewall, DMZ, reverse proxy, WES, network based WAF, load balancing
- Monitor network: Wireshark, MRTG, Nmap, Nagios
- Implement NIDS/NIPS
- Implement tarpits (LsBrea, Netfilter)
- Implement HIDS/HIPS
- Implement host based WAF and Spam-Filter
- Classify data
- Implement DLP
- Implement honeypots
- Make research on external legislative requirements

Glossary:
TTP: Tactics, Techniques and Procedures
IoC: Indicator of compromise
APT: Advanced persistent threat
SIEM: Security incident and event management
IAM: Identity access management
WES: Web entry server
NIDS: Network intrusion detection system
NIPS: Network intrusion prevention system
HIDS: Host intrusion detection system
HIPS: Host intrusion prevention system
DLP: Data loss prevention