Some time ago among security researchers, it was very “fashionable” to find improperly configured AWS cloud storages with various kinds of confidential information. At that time, I even published a small note about how Amazon S3 open cloud storage is discovered.

However, time passes and the focus in research has shifted to the search for unsecured and exposed public domain databases. More than half of the known cases of large data leaks over the past year are leaks from open databases.

Today we will try to figure out how such databases are discovered by security researchers...

Required disclaimer: this article does not address and does not raise legal and ethical issues related to the search and identification of publicly available databases. All the information below should be considered solely as educational.

I will not “discover America” if I say that the main tools for searching open databases are the specialized search engines Shodan.io, BinaryEdge.io and Censys.io. I’ll just make a note that these are not free services, and you have to pay for access to the full search results (in the case of Shodan it is $59/month, BinaryEdge — $50/month, and for Censys — $99/month for the minimum basic packages). Free versions of search engines severely limit the number of results in the issuance.

In addition to the classic Google-like search experience using the search function, these search engines provide the ability to connect to them through the API. The lucky owner of a paid subscription will receive an API key. This allows one to significantly facilitate the task of clearing the heap of search rubbish. I have provided a list of useful scripts and search automation programs that use API keys at the end of this article.

Let's take a look at the example of the Shodan search engine: how to find open MongoDB and Elasticsearch databases.

The simplest and most obvious request that begs an attempt is “MongoDB”:

From a screenshot above, we can see that this request returned all indexed MongoDB servers (on default port 27017). Moreover, most of them (the first three in the screenshot ) will be closed databases requiring account credentials. It’s not exactly what we are interested in finding. More precisely, it is not interesting at all.

Let's slightly complicate the query using the search filters “all:"mongodb server information" all:"metrics"”:

The results look much better. All found databases were freely available at the time when the search engine indexed them. It is very likely that we will be able to connect to the specified IP addresses using any manager for MongoDB (for example, NoSQL Manager for MongoDB or Studio 3T for MongoDB).

To find unauthenticated MongoDB instances that also act as file storage servers, you may try to use the query “port:27017 -all:"partially" all:"fs.files"”

One can limit the search to any country. Let's say to look for the open MongoDB’s in China: (query “all:"mongodb server information" all:"metrics "country:"cn"”):

To search for Elasticsearch open databases, it is convenient to use the query “port:"9200" all:"elastic indices"”:

To impose more complex filters (restrictions on the size of the database, the date of the server in the index, etc.) on the search results, one can use specialized scripts and programs. To do this, one will need paid access and an API key, as I wrote above.

Here is a small list of links that might be useful:

• cli.shodan.io is the official Shodan console.
• github.com/woj-ciech/LeakLooker is a python program for searching open databases (MongoDB, CouchDB, Elasticsearch, etc.).
• lampyre.io is a very powerful (and still very buggy!) graphical interface for searching and visualizing what you find. It has built-in templates for MongoDB search and much more. Its in beta testing with all the consequences that implies...