Skip to content

Commit

Permalink
(fix/tools)proxy usage with socks & http (#69)
Browse files Browse the repository at this point in the history
* (fix/proxy)using GetHttpTransport() to configure http settings

still a some work to do in order to have Proxy set for every tool if
it is specified by the user, but having a helper function to easily
fetch http.Transport with the correct values is a good start

(also fixed some english spelling mistakes)

next step is to add the proxy to the configuration of bigger projects,
like httpx & nuclei. AFAIK httpx does not use ProxyFromEnvironment in
order to set the proxy, which means we probably will have to set it
manually. This can be annoying as this is not a mandatory setting

will also probably need to rework dorks through a cleaner interface,
because using exec.Command() prevents us from manually setting a proxy.

* (mod/build)adding rule to dynamically build Yelaa

To run Yelaa with Proxychains, one needs to have a dynamically compiled
program. This is because Proxychains uses LD_PRELOAD tricks to hook to
connect() and set the proxy, which is not possible with a statically
program such as one compile with gc (standard go compiler)

the script used to compile is still very much a WIP and other options
could be explored instead:
    - use a TUN interface (https://github.com/nicocha30/ligolo-ng)
    - TUN to SOCKS (https://github.com/russdill/tunsocks)
    - iptables tricks (https://github.com/n1nj4sec/pr0cks) but this
      would not work very well with docker

* (mod/gowitness)setting proxy option in chrome

- dorks was broken with socksproxy so the option for that was removed.

scan command is broken, further testing is required

* (fix/tools)manually setting proxy for most interfaces

* (mod/docs)improving docs regarding make dynamic rule

* (mod/main)removing YELAA_PROXY env value usage

* (mod/ci)bumping tool version & improving docs a little
  • Loading branch information
djnnvx authored Nov 18, 2022
1 parent 79f1f05 commit 7f54f58
Show file tree
Hide file tree
Showing 17 changed files with 187 additions and 36 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ target.txt
# Build files
dist/
Yelaa
DynYelaa
nuclei-templates/

# Out files
Expand Down
7 changes: 6 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@

TARGET = Yelaa
DYN_TARGET = DynYelaa
SRC = main.go

GROUP_ID = $$(id -g)
Expand All @@ -24,6 +25,10 @@ docker: ## Builds a docker image from source
--build-arg GROUP_ID=$(GROUP_ID) \
.

.PHONY: dynamic
dynamic: ## Builds a dynamically linked binary (if you really need to use Proxychains)
@./scripts/compile_dyn.sh

.PHONY: clean
clean: ## Cleans up the project
rm -f $(TARGET)
rm -f $(TARGET) $(DynYelaa)
59 changes: 45 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,9 @@ wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb

sudo apt install ./google-chrome-stable_current_amd64.deb

wget https://github.com/CMEPW/Yelaa/releases/download/v1.6.2/Yelaa_1.6.2_Linux_x86_64.tar.gz
wget https://github.com/CMEPW/Yelaa/releases/download/v1.7.0/Yelaa_1.7.0_Linux_x86_64.tar.gz

tar -xvf Yelaa_1.6.2_Linux_x86_64.tar.gz
tar -xvf Yelaa_1.7.0_Linux_x86_64.tar.gz
./Yelaa -h
```

Expand All @@ -66,12 +66,20 @@ You can run `Yelaa create -c <client> -s <PathToSharedFolder>`

`Yelaa scan -t <PathToTargetFile>`

## Use http proxy
## Use http / socks proxy

`Yelaa scan -p http://localhost:8080 -target ./targets.txt`
```bash
# using a http proxy
Yelaa scan -p http://localhost:8080 -target ./targets.txt`
# or, socks5 proxy
Yelaa scan -p socks5://localhost:9050 -target ./targets.txt`
```

>Flag `-k` is available to skip tls configuration
>Please prefer using socks5 as much as possible, as socks4 can fail depending on your go version
## How to run osint on a domain

`Yelaa osint -t ./targets.txt -p http://localhost:8080 --path /tmp`
Expand All @@ -89,11 +97,16 @@ To run osint command on several domains run `Yelaa osint -t targets.txt`

## Low fruits : Infrastructure Penetration Testing

`nmap -T4 -Pn -p 80,443,8080,8443 --open -oA EvilCorp-24 192.168.1.0/24`
```bash
# run scan on ports 80, 443, 8080 & 8443
nmap -T4 -Pn -p 80,443,8080,8443 --open -oA EvilCorp-24 192.168.1.0/24

then `cat *.gnmap | grep -i "open/tcp" | cut -d " " -f2 | sort -u > web-targets.txt`
# fetch tcp open ports & put them in web-targets.txt
cat *.gnmap | grep -i "open/tcp" | cut -d " " -f2 | sort -u > web-targets.txt

Finaly `./Yelaa checkAndScreen -t ./web-targets.txt`
# run check-and-screen to quickly map infra
./Yelaa checkAndScreen -t ./web-targets.txt
```

## Help

Expand All @@ -112,27 +125,45 @@ Usage:
Available Commands:
checkAndScreen Run httpx and gowitness
help Help about any command
osint Run subfinder, dnsx and httpx to find ips and subdomains of a specific domain
scan It will run gobuster and store logs in .yelaa (by default)
completion Generate the autocompletion script for the specified shell
help Help about any command
osint Run subfinder, dnsx and httpx to find ips and subdomains of a specific domain
scan It will run Nuclei templates, dirsearch and more.
Flags:
-c, --client string Client name
--dry-run Run in dry-run mode
-e, --excludedType string excluded type
-h, --help help for create
-k, --insecure Allow insecure certificate
--nuclei Enable nuclei with the command
--path string Output path (default "/home/$USER/.yelaa")
-p, --proxy string Add HTTP proxy
--rate-limit int32 Rate limitation for nuclei and gobuster (default 100)
-s, --shared string path to shared folder
--path string Output path (default "/home/$USER/.yelaa")
--nuclei Run nuclei
-w --wordlist string Path to wordlist that's gobuster will use for scan command (default "yelaa.txt")
Use "create [command] --help" for more information about a command.
All temporary file have been succesfully removed
```

> This script will create a default structure using `create` command, as well as a cherytree database with payloads for external testing and useful commands for internal testing
## run with Proxychains

> this is not the recommanded way to use a proxy! You can just specify a proxy with the `-p` option!
If you *must* run Yelaa through Proxychains, it is possible but will require a bit of tweaking.
The reason for that is that Yelaa is statically compiled, and `Proxychains` uses `LD_PRELOAD` tricks to set a proxy.
You will have to compile Yelaa dynamically, using `gcc-go` (you will have to [install it yourself](https://go.dev/doc/install/gccgo) before compiling):

```bash
git clone https://github.com/CMEPW/Yelaa.git

cd Yelaa

make dynamic
```

# Contributors

| [<img src="https://github.com/darkweak.png?size=85" width=85><br><sub>Darkweak</sub>](https://github.com/darkweak) | [<img src="https://github.com/jenaye.png?size=85" width=85><br><sub>Mike Houziaux</sub>](https://github.com/jenaye) | [<img src="https://github.com/jarrault.png?size=85" width=85><br><sub>Julien</sub>](https://github.com/jarrault) | [<img src="https://github.com/TomChv.png?size=85" width=85><br><sub>Tom Chauveau</sub>](https://github.com/TomChv) | [<img src="https://github.com/bogdzn.png?size=85" width=85><br><sub>bogdan</sub>](https://github.com/bogdzn)| [<img src="h[ttps://github.com/bogdzn](https://github.com/VidsSkids.png?size=85" width=85><br><sub>VidsSkids</sub>]([https://github.com/bogdzn](https://github.com/VidsSkids))
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ require (
github.com/OJ/gobuster/v3 v3.1.0
github.com/apex/log v1.9.0
github.com/blang/semver v3.5.1+incompatible
github.com/bogdzn/gork v0.0.3
github.com/common-nighthawk/go-figure v0.0.0-20210622060536-734e95fb86be
github.com/fatih/color v1.13.0
github.com/go-playground/validator/v10 v10.10.0
Expand Down Expand Up @@ -52,7 +53,6 @@ require (
github.com/bits-and-blooms/bitset v1.2.0 // indirect
github.com/bits-and-blooms/bloom/v3 v3.0.1 // indirect
github.com/bluele/gcache v0.0.2 // indirect
github.com/bogdzn/gork v0.0.3 // indirect
github.com/c4milo/unpackit v0.1.0 // indirect
github.com/caddyserver/certmagic v0.15.2 // indirect
github.com/chromedp/cdproto v0.0.0-20210728214956-1fab41c4e0b7 // indirect
Expand Down
19 changes: 18 additions & 1 deletion helper/helper.go
Original file line number Diff line number Diff line change
@@ -1,10 +1,27 @@
package helper

import "os"
import (
"net/http"
"net/url"
"os"
)

var YelaaPath = GetHome() + "/.yelaa"

func GetHome() (home string) {
home, _ = os.UserHomeDir()
return
}

func GetHttpTransport() (*http.Transport) {
var proxy = os.Getenv("HTTP_PROXY")
url, err := url.Parse(proxy)

if proxy != "" && err == nil {
return &http.Transport{
DisableKeepAlives: true,
Proxy: http.ProxyURL(url),
}
}
return &http.Transport{}
}
18 changes: 12 additions & 6 deletions main.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ import (
"fmt"
"io"
"io/ioutil"
"net/http"
"os"
"os/exec"
"strings"
Expand Down Expand Up @@ -66,13 +65,15 @@ func loadTargetFile() *FileScanner {
}

func readFile() {
http.DefaultTransport.(*http.Transport).TLSClientConfig = &tls.Config{InsecureSkipVerify: insecure}
transport := helper.GetHttpTransport()
transport.TLSClientConfig = &tls.Config{InsecureSkipVerify: insecure}

var toolList []tool.ToolInterface
toolList = append(toolList, &tool.Robot{}, &tool.Sitemap{})

gb := tool.GoBuster{}
cfg := make(map[string]interface{})
cfg["proxy"] = proxy
cfg["scanPath"] = scanPath
cfg["rateLimiter"] = rateLimit
cfg["wordlist"] = wordlist
Expand Down Expand Up @@ -150,14 +151,16 @@ func folderNameFactory(names ...string) []folder {
}

func checkProxy() {
os.Setenv("HTTP_PROXY", proxy)
os.Setenv("HTTPS_PROXY", proxy)
os.Setenv("HTTP_PROXY", proxy)
os.Setenv("HTTPS_PROXY", proxy)

if proxy != "" {
color.Cyan("Proxy configuration: %s", proxy)
} else {
color.Cyan("No proxy has been set")
}


}

func createOutDirectory() {
Expand All @@ -176,7 +179,6 @@ func scanDomain(domain string) {
dorksCfg := make(map[string]interface{})

dorksCfg["outfile"] = scanPath + "/dorks.txt"
dorksCfg["proxy"] = proxy

dorks.Configure(dorksCfg)
dorks.Info(domain)
Expand All @@ -197,6 +199,7 @@ func scanDomain(domain string) {
sf := tool.Subfinder{}
configuration := make(map[string]interface{})
configuration["filename"] = subdomainsFile.Name()
configuration["proxy"] = proxy
sf.Info("")
sf.Configure(configuration)

Expand Down Expand Up @@ -257,6 +260,7 @@ func scanDomain(domain string) {
httpxConfig := make(map[string]interface{})
httpxConfig["input"] = scanPath + "/domains.txt"
httpxConfig["output"] = filepath
httpxConfig["proxy"] = proxy
httpx.Info("")
httpx.Configure(httpxConfig)

Expand All @@ -268,6 +272,7 @@ func scanDomain(domain string) {
gwConfig := make(map[string]interface{})
gwConfig["file"] = filepath
gwConfig["scanPath"] = scanPath
gwConfig["proxy"] = proxy

gw.Info("")
gw.Configure(gwConfig)
Expand All @@ -281,7 +286,7 @@ func scanDomain(domain string) {
}

func main() {
version := figure.NewColorFigure("Yelaa 1.6.2", "", "cyan", true)
version := figure.NewColorFigure("Yelaa 1.7.0", "", "cyan", true)
version.Print()

var cmdScan = &cobra.Command{
Expand Down Expand Up @@ -353,6 +358,7 @@ func main() {

gw := tool.Gowitness{}
gwConfig := make(map[string]interface{})
gwConfig["proxy"] = proxy
gwConfig["scanPath"] = scanPath
gwConfig["file"] = filepath
gw.Info("")
Expand Down
58 changes: 58 additions & 0 deletions scripts/compile_dyn.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
#!/usr/bin/env bash

set -eou pipefail

#
# This script is a work-around to use gccgo with reflect2
# (which is a dependency of protobuf, so it's can of hard for us to
# fix this manually)
#
# Reference: https://github.com/modern-go/reflect2/issues/21
#
# I don't like this either, but this has been the only way to make proxychains
# work, because it relies on LD_PRELOAD to set proxies, so we should use gccgo
# to resolve libc dynamically
# https://github.com/golang/go/issues/31772#issuecomment-488322661
# https://github.com/Jguer/yay/issues/429#issuecomment-393661439
#

PKG_PATH="${GOPATH}/pkg/mod/github.com/modern-go/[email protected]"
FILENAME="unsafe_link.go"

PROGRAM_NAME=DynYelaa

compile () {
echo "[+] Running compile with gccgo"
go build -compiler gccgo -o ${PROGRAM_NAME}
}

backup_program_files () {
echo "[+] Making backups of soon-to-be modified file"

sudo cp -v "${PKG_PATH}/${FILENAME}" "/tmp/${FILENAME}"
mv ~/.cache ~/.cache.bak
}

reset_cache () {

echo "[+] Resetting file"
sudo mv -v "/tmp/${FILENAME}" "${PKG_PATH}/${FILENAME}"

mv ~/.cache.bak ~/.cache
}

replace_files () {
echo "[+] Fixing reflect.unsafe_New call in ${PKG_PATH}/${FILENAME}"

sudo sed -i 's/go:linkname unsafe_New reflect.unsafe_New/go:linkname unsafe_New reflect.unsafe__New/' "${PKG_PATH}/${FILENAME}"
sudo sed -i 's/go:linkname unsafe_NewArray reflect.unsafe_NewArray/go:linkname unsafe_NewArray reflect.unsafe__NewArray/' "${PKG_PATH}/${FILENAME}"
}

backup_program_files

replace_files

reset_cache

echo "[+] Successfully generated ${PROGRAM_NAME}:"
file "${PROGRAM_NAME}"
16 changes: 12 additions & 4 deletions tool/dirsearch.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,22 @@ package tool

import (
"fmt"
"os"
"os/exec"
)

func Dirsearch(url string) {
args := "-u"
args2 := url
out, err := exec.Command("dirsearch", args, args2).Output()
func run(url string) ([]byte, error) {
proxy := os.Getenv("HTTP_PROXY")

if proxy != "" {
proxyCmd := fmt.Sprintf("--proxy=%s", proxy)
return exec.Command("dirsearch", "-u", url, proxyCmd).Output()
}
return exec.Command("dirsearch", "-u", url).Output()
}

func Dirsearch(url string) {
out, err := run(url)
if err != nil {
fmt.Printf("%s", err)
}
Expand Down
5 changes: 1 addition & 4 deletions tool/dorks.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ import (

type Dorks struct{
outfile string
proxy string
userAgent string
extensions []string
}
Expand All @@ -29,14 +28,12 @@ func (d *Dorks) Configure(c interface{}) {
d.extensions = defaultExtensions
d.userAgent = defaultUserAgent
d.outfile = c.(map[string]interface{})["outfile"].(string)
d.proxy = c.(map[string]interface{})["proxy"].(string)
}

func (d *Dorks) Run(url string) {
opts := &dorks.Options{
Outfile: d.outfile,
AppendResults: false,
Proxy: d.proxy,
AppendResults: true, /* we could be running this in a loop, should not erase former results */
Extensions: d.extensions,
UserAgent: d.userAgent,
Target: url,
Expand Down
3 changes: 3 additions & 0 deletions tool/gobuster.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ type GoBuster struct {
optDir *gobusterdir.OptionsDir
opts *libgobuster.Options
scanPath string
proxy string
}

func (s *GoBuster) Info(website string) {
Expand All @@ -39,6 +40,8 @@ func (g *GoBuster) Configure(c interface{}) {
fmt.Println(err)
}
}

g.proxy = c.(map[string]interface{})["proxy"].(string)
g.optDir = gobusterdir.NewOptionsDir()
g.optDir.StatusCodesBlacklistParsed.Add(404)
g.optDir.NoTLSValidation = true
Expand Down
Loading

0 comments on commit 7f54f58

Please sign in to comment.