Intro

Now that we’ve covered some basics of what OSINT is, why we use it, and how it might benefit us, let us look at the core of what makes up our collective intelligence effort. Do note that even though we should be familiar with this, every analyst should develop their own techniques, methods, and even tools, depending on the case they’re investigating. Think of what we’re describing below as a loose guideline that can be used in your investigations.

Also, please note that the Intelligence Cycle, as described below, is of a broader scope, and doesn’t necessarily pertain to OSINT investigations exclusively. However, from our perspective it is vital to be at least familiar with it, which is why we chose to dedicate that much space to it.

Intelligence Cycle

The Intelligence Cycle is the process of compiling raw data into intelligence that can be used to make decisions, be it for LE use, or for business driven purposes. In its nature, the Intelligence Cycle is cyclical (hence Intelligence Cycle) – meaning that what we’ve discovered previously can influence the following routes of our investigation. The goal here is to remain open to new information, and understand that it can impact the current state of affairs in our investigation.

The Intelligence Cycle consists of five parts: 

1. Planning and Direction
2. Collection
3. Processing
4. Analysis and Production
5. Dissemination

Planning and Direction

This part involves the management of the whole investigation, from identification of our intelligence needs, to delivering of the said intel. It is both the beginning and the end of the cycle, because it involves defining our needs (planning), the end, because once finished our new intel can produce new informational needs. This is due to the fact that our intel needs to work hand-in-hand with our decision making, which might change once we reach the end – thus leaving us at the beginning of the cycle once more.

Collection

To collect intel effectively, we need to have a good plan that we’ll stick to, as well as some sort of direction. Since in this phase we’re collecting raw data, open sources can be a treasure trove for us here; also, in the context of a pure OSINT investigation, here we would implore the analyst to pay most of their attention, and deploy their critical thinking ability as much as possible. Data can be extremely volatile, and we need to understand not only the data points itself, rather we should visualize the broader picture. (Once more, the Intelligence Cycle is a bit more broader, and goes beyond only open source data)

Processing

In this step, we convert the raw data that we’ve gathered into a format an analyst can work with. This entails managing our information, through whatever techniques we may deem necessary for our particular investigation. We reduce the data, arrange, and process it in such a way that it can be of use to the one who would be consuming it.

It follows that this step will differ greatly if we are, for example, processing our data for a LEA, or if its an investigation where we would be the consumer of the said information – if we are maybe gathering intel for a penetration test; at least before compiling the report for our client.

Analysis and Production

Analysis and production pertains to us converting all of the information that we’ve processed, into a finished product. This intel is evaluated, integrated, and further analyzed. The data is integrated into one coherent whole, what was evaluated is put in context, and then produced into a finished piece of intelligence – which includes assessments, and implications of the intel, in that particular context.

Dissemination

In this final phase, we distribute our intel to the consumer, the same ones who initiated the process with their intelligence needs and requirements. Then, based on the information, the consumer would make their decisions, which may trigger the Intelligence Cycle again.

Thoughts, conclusions

It is apparent that this type of approach is generally more geared toward LEA’s, or some businesses, but, as an aspiring OSINT analyst, we should be aware of how these things are usually done. There’s a lot of things for us to unpack here, and even though we might not use or need to follow the exact same steps, we will, however, still act somewhat in convergence with the model above.

The main takeaway, for us, is the fact that this kind of approach has a great impact on how we can further use our own critical thinking and deductive skills, since critical thinking is the most important skill an OSINT analyst needs to possess – in our opinion. That is, the ability to think rationally about the topic, in an organized way, so that we can best understand the connection of the facts that are presented to us.

For example, we should always look to define our problems and/or questions as precisely as we can. We also need to find different sources – in order to understand different points of view. Further, we should evaluate the reliability of said sources, understand if they’re biased, and if that’s the case, then we would be interested in how’s and why’s.

Once we’ve weeded out some of those crucial questions, and further crystallized our picture, we would try and understand what’s most important of the facts that we’ve gathered. Finally, once we do all of that successfully, we need to know how to present this coherently, to whomever might be the party to which our investigation refers to.

With all this in mind, of course every analyst’s process will differ, but the way in which we go on about our investigation, should be grounded around some of the same core principles. Remember, your greatest and most important tool is your ability to rationalize, analyze, connect the dots, and make good deductions based on all of that – your critical thinking ability.

OSINT Framework

Before concluding our article, we would like to mention one more thing – the OSINT Framework.

This is a web-based platform, which bundles a lot of different OSINT tools – on many different themes, such as: IP address, Images, Social Networks, People Search Engines, Public Records, Metadata, Dark Web, and many more.

Most of them are free to use, but there’s a number of tools that are subscription-based. Nevertheless, this can be a great starting point for your investigation, and is something every OSINT analyst should be well aware of, in our opinion.

OSINT Framework Homepage

Conclusion

To conclude, we’d just like to mention that the idea behind ‘teasing’ with the OSINT Framework in this article is due to the fact that our next article will focus on some of the tools one might use in their investigation, so we felt it was a good inclusion and a natural transition; at least now that we’ve laid some groundwork, and explained, albeit briefly, some of the core intelligence gathering ideas.

As we will see, there’s a myriad of tools out there, and everybody has their own preferences, but the ideas behind them are generally nested around their theme/functionality.

Lastly, here’s another teaser for you, before we go delving into the tools in our next article!

How OS Fingerprinting  Works: Fundamentals You Need to Know

Intro

In this article – the last in our John the Ripper series – we would like to focus on how we can use John to crack SSH keys, as well as mention some basics of Custom Rules.

SSH

What is SSH? When do we use it (or should)? How does it work, and what are some encryption techniques/technologies that SSH has to offer?

Let’s answer all of these questions briefly (it is a very big topic), before delving further into how john can leverage some of its functionalities to crack the SSH private key password of the id_rsa files.

SSH stands for Secure Shell, and is a remote administration protocol, which gives us the ability to access, control, or modify our remote infrastructure (usually servers) over the Internet. You might want to remote to your clients server to troubleshoot something, or to deploy some code.

Historically, SSH was created as a replacement for the much more insecure protocol called Telnet, which, even though with the same purpose, doesn’t offer encryption. You can see why that might make some of us feel quite awkward. SSH encrypts all of our communication to and from the remote server, by the virtue of encryption. With SSH we can authenticate a remote user, for example.

To use SSH, we can simply pull up the terminal (for MacOS/Linux) and type:

ssh <username>@<ip_address> -p(port_number)

Where the username is the name of the user we wish to connect as, and the IP address being that of our server we are connecting to. For Windows we can use a SSH client, the most known one being PuTTY.

For example, if we were to connect as a user called john to our remote server at 184.121.23.43 at the default port (for SSH its port 22), we would give a command like this:

ssh john@184.121.23.43 -p22

Regardless of our platform, once we’ve issued our command, we will get a prompt asking for a password for the user we specified, in order to authenticate us. If the credentials are correct, we will be shown a command-line, that of our server we just got into.

SSH and John the Ripper

As we’ve already mentioned, we can use john to crack private key passwords of our id_rsa files. If our target has configured key-based authentication – which just means they are using their private key – id_rsa – as their key to authenticate against the server and to log in using SSH. Since this will generally require a password, we can once again use John to help us crack that password, so that we can authenticate over the SSH (by the usage of the said key).

Another tool (as zip2john, and rar2john previously – sound familiar?) john leverages, is a tool called ssh2john. The logic remains the same – ssh2john converts the id_rsa key to a hash that John can work with. The syntax is virtually the same as before:

ssh2john [id_rsa_file] > [output_file]

ssh2john – command to call our converter tool

id_rsa_file – path to our file that we want to convert to a hash

output_file – here, we will store our output e.g. the hash that we’ve created

One small thing of note, before we look at our example. If your terminal tells you that ssh2john can’t be found (command not found – meaning ssh2john is not installed – like in the image below)

Please note that you can still use ssh2john.py, which is basically the same thing, wrapped inside a Python script. Usually, ssh2john.py is located in /opt/john/ssh2john.py or, in case you’re using Kali, you can find it in /usr/share/john/ssh2john.py. Just remember to invoke your Python scripts by adding python/python3 to your command line first. (as shown in the image below)

This also brings us to our example.

In order to do the cracking, we’ve first created a new private/public key-pair using ssh-keygen (image below)

(Spoiler alert! We’ve used the passphrase banana)

All that’s left now is to do some john magic.

First, we run our Python version of the ssh2john conversion tool – as shown below (which is the same image as above)

Simply, we’ve asked Python to run the script called ssh2john, which can be found in the /usr/share/john/ssh2john.py path… again, if you’re not on Kali, this would be /opt/john/ssh2john.py, and then we’ve given the path to our newly created (banana-protected) private key – /root/.ssh/id_rsa – which we’ve redirected to an output file on our Desktop, called KeyHash.txt.

Now we are ready, and should have all we need in order for John to crack our private key password for us.

We invoke John, using our trusty rockyou.txt wordlist, and let it do its thing:

 

Lo and behold, 29 seconds after, John has returned to us with the correct output – banana, cracking our password successfully!

Custom Rules

Similarly to the single crack mode that we’ve covered in part 2 of our series (word mangling, or variations of a word, where we change the letters to capital letters, numbers, etc.) we can also define our own sets of rules in similar fashion. John will then use our newly created rules to create passwords. This can be quite useful if we know (or suspect) the password structure of whatever it is that we’re attacking.

With this we can integrate capital letters, numbers, symbols… same as for the single crack mode. Also, this can prove to be rather useful for us, since organizations sometimes enforce password policies in order for them to be a bit less susceptible to dictionary attacks.

This is exactly what an attacker might leverage to their advantage! As we all know people tend to make similar passwords, or even reuse them, and adding numbers and capital letters, or symbols can make it so they meet the password policy’s requirement (complexity). Still, Babyblue1! is not an example of a secure password by any means!

So, if an attacker knew about the password structure, used a bit of Social Engineering on the target they’ve picked (some employee of the company perhaps), they could then easily connect the dots and compromise the system – gain a foothold into your now compromised organization.

Password rules are usually located in the /etc/john path, in a file called john.conf. Another path could be /opt/john.

To create our rule, the first line is used to create a name for the rule, which we can later invoke with John. It looks something like this: 

[List.rules:Babyblue]

Then, we need to use a regex style pattern in order to define our rule further:

A0 – prepends the word with characters we defined

c – capitalization of the character (position based!)

Az – appends the word with any characters we defined

u – convert to uppercase

Now we just need to decide where and what we want to be changed. To define what’s going to be prepended or appended, we put that in square brackets [] – in the order of usage!

We end up with something similar to this:

cAz”[0-9] [!@%$]”

After that, all that’s left is to add our rule to our usual command, by adding this flag: –rule=Babyblue.

We would end with a command like this:

john –wordlist=/usr/share/wordlists/rockyou.txt –rule=Babyblue target_file_path

Of course, there are many resources out there, and we would suggest first checking out these two, if all this talk about custom rules has piqued your interest.

Conclusion

Some finishing thoughts before we close out this series about John the Ripper. As we’ve seen from some of our examples and stuff that what was mentioned in the series, John offers a lot of flexibility and versatility, but, as always, in order to leverage this great tool to its maximum potential, there’s a lot of ground to be covered – this does not mean you need a PhD in Cryptography, of course, just a lot of trial and error!

We wish you happy (& safe) password cracking!

Intro

It should come as no surprise that John can also deal with .zip and .rar archives. John does this by leveraging the zip2john and rar2john utilities, built in the tool, so that it can ingest something that it will know how to use. The syntax is pretty much the same, and by now, you should be quite familiar with it; still, we will use this article to show some examples of how we can make our .zip and .rar archives John-ready. As we will see, this is akin to the unshadow tool we’ve used previously.

zip2john

As mentioned, similar to unshadow, John has the inbuilt tool that’s called zip2john, which we use to convert our target .zip archive into a format john will know what to do with, and, we hope, crack successfully.

The basic syntax looks something like this:

zip2john [target_zip_file] > [output_file]

Flags:

target_zip_file – this is the path to our password protected protected .zip archive

> – greater than sign which redirects our command results to a specified output file

output_file – in this file we store our output

So, our command will look something like this:

zip2john target_archive.zip > zip_hash.txt

Once we’ve successfully obtained the zip_hash.txt output file, we simply supply it to John. And yes, we can use the wordlists too. Thus, we just had a couple of extra steps before we return to using John as we’ve already learned previously.

More simply put, we can say to John, something like this:

john –wordlist=/usr/share/wordlists/rockyou.txt zip_hash.txt

As you can see, this is something that we’ve already learned, and we’ve just used the zip2john utility to prepare our archive for John to work with. 

Let’s quickly cover rar2john next, and then we will go over some examples.

rar2john

The same as zip files, rar can also compress various files and folders. It does so by using the Winrar archive manager.

We use it in the same way as zip2john. First, we use rar2john to make the .rar archive ready for John – by obtaining its hash, then we supply the said hash to John to try and crack it.

The syntax is the same as for zip2john:

rar2john [target_rar_file] > [output_file]

Flags:

target_rar_file – this is the path to our password protected protected ..rar archive

> – greater than sign which redirects our command results to a specified output file

output_file – in this file we store our output

It will look something like this:

rar2john rar_archive.rar > rar_hash.txt

Now we just use John, the way we’ve learned, giving it our rar_hash.txt file:

john –wordlist=/usr/share/wordlists/rockyou.txt rar_hash.txt

Examples

Now that we’ve covered some basic stuff, let’s look at some examples.

We have a .zip archive, called testZip1.zip with three files inside, called pass.txt, sop.txt, and supersecret.txt.

To digress for a bit: it would be terrible if we saw something like this on our test, as an attacker. The naming convention here in our article is there for our convenience and illustrative purposes. Nobody should have a file called sop.txt (which usually stands for standard operating procedure). Pass.txt and supersecret.txt even less so, for obvious reasons.

Going back to our .zip cracking, we give John a command asking it to make an output file it can understand, and try to crack it. (image below)

We just gave John our output file (in this case test1.hashes), and it cracked our password. Note that you can make your output file to be of any format, for further processing/manipulation. Use what makes the most sense for your purposes.

Since we have our password, we show it with John, and try to open our password protected archive, which prompts us:

Finally, we have our cracked .zip archive, and contents opened, of one of the files:

Let’s look at some more examples.

We password protect our .rar archive, called safe.rar, as we can see – password is password1. Inside, we have put two files, called pass.txt, and pass2.txt, respectively. We now need to make something John can use, out of our .rar archive:

Now we ask John to crack our new file, called rar_cracked.hash, for demonstration, we passed no arguments/options to John first.

When we ask John to crack something, without giving other arguments, it will go through the default modes, with their default settings. (That’s why it started with single crack mode first, in the image above)

Since we know our password is really weak, and we don’t want to wait that much, we use our trusty rockyou.txt wordlist:

Our wordlist mode works immediately, giving us the password we were looking for – password1, as shown above.

We then try to open our archive, finally:

As we can see, we get a popup saying we need to enter our password to access safe.rar, which we type in:

Et voila! We have managed to access the .rar archive:

From the image above – two files, called pass.txt and pass2.txt, as mentioned previously – and their contents.

Conclusion

We’ve seen how we can use John to crack password protected .zip and .rar archives, and how it’s just one extra step after what we’ve already covered previously. Please note, though, that we were using just the rockyou.txt wordlist in our examples, and there are many wordlists out there, of different sizes (which can definitely speed up your attempts), purpose/type (why stop at passwords? We can also have URLs, web shells, fuzzing payloads, etc). Personally, I like to start off with the shortest wordlist that suits my particular need, as it is simply the fastest way. After that, if I don’t crack anything, I can easily switch to some larger lists.

One awesome resource would be the Openwall wordlists found on Openwall website – which is John the Ripper’s original website. There you can find some publicly downloadable lists, as well as paid ones, which can dramatically increase your password recovery potential, as that particular collection has 20+ languages, over 40 million entries, and also has pre-applied mangling rules (in this way you can do other likely password variations – adding digits instead of words, capitalization, etc.)

Before finishing, we would like to add that the best way to get these materials to ‘stick’ is to go and try for yourself. So, go fire up a VM, make some archives, add some files to it, password protect it, and attempt to crack it! (Once you master the easy ones, like in our examples, it’s time to start attacking some more complex passwords, and that’s where the greatest fun begins)

The public’s favorite government agency, CISA (not the CIA) has recently added five new vulnerabilities to its Known Exploited Vulnerabilities Catalog, the living, breathing, exponentially growing list of vulnerabilities that have seen active exploitation in the wild.