[RFC] New scanner interface

[sundowndev]

This issue is a RFC about major architectural changes in the upcoming next versions of PhoneInfoga.

Context

It's been a while that PhoneInfoga stopped working properly due to scraping limitations. I knew since the beginning that this project couldn't work without constant maintenance. The goal of this new version is to bring a new set of features to the existing project and stop relying on scraping to gather information about phone numbers.

High-level roadmap

• More API usage and less scraping
• A new scanner API and abstraction layer. Creating a new scanner will be easier.
• Concurrency mode: scanners should be able to run in concurrency mode to enhance the user experience.
• Bring new commands to interact with phone numbers (e.g. vCard file generation)
• Support HLR lookups through different APIs
• Support profile lookup on messaging apps such as Whatsapp, Telegram, ...
• Support for plugins: user can provide its own scanners

That means the tool will no longer rely on free resources and the user will need to specify one or more API keys in order to use the tool. Each scanner will become optional so the user can choose which one should be used in a scan.

Usage examples :

# numverify scanner is disabled
$ phoneinfoga scan +3367823xxxx --ignore numverify

# A plugin is used for this scan
$ phoneinfoga scan +3367823xxxx --plugin ./custom_scanner.so

# numverify scanner is enabled, format is JSON, send output to a file
$ NUMVERIFY_API_KEY=<key> phoneinfoga scan +3367823xxxx -f json -o result.json

Definitions

• Payload : the user input to be scanned.
• Scanners : logic that fetch, search or generate information about the payload.
• Suppliers : the data access layer to request remote APIs (e.g.: Google API, Numverify API, ...). It's primarily used by scanners.
• Output : the output to be used to print the scan result (console, json, ...). It requires a path so it supports both files and stdout.

Backward compatibility

We MUST break as less as possible so consider the following :

• We'll deprecate -n option for the scan command but continue to support it
• Recon command is already deprecated, it will be entirely removed
• REST API will follow the same OpenAPI specs and new routes will use the v2 namespace

Scanners

This is a non-exhaustive list of scanners we could implement in the future. Each of them still requires its own investigation to better understand how it works and how it can bring value to Phoneinfoga users. Some scanners may require a rate-limiting mechanism (e.g. Pipl).

• googlecse
• neutrino
• hlrlookup
• infobel
• instagram
• telegram
• whatsapp
• pipl
• dehashed
• yandexsearch
• bingsearch
• twilio

References

Social medias: Facebook, Twitter, Weibo, TikTok, Instagram, LinkedIn, VK, Snapchat, YouTube, Periscope
Search engines: Google, DuckDuckGo, Yandex, Bing, Yahoo, wayback machine, archive.ph

• https://www.itu.int/dms_pub/itu-t/oth/02/02/T020200004A0002PDFE.pdf
• https://www.secjuice.com/phone-numbers-investigation-the-open-source-way/
• https://tools.ietf.org/html/rfc6350

[Robertrabbit]

🐇