Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workaround for cloudflare protected websites #1715

Open
Slasar41 opened this issue Oct 10, 2024 · 10 comments
Open

Workaround for cloudflare protected websites #1715

Slasar41 opened this issue Oct 10, 2024 · 10 comments

Comments

@Slasar41
Copy link
Collaborator

Slasar41 commented Oct 10, 2024

Preparation

  • Install python. Enable "Add python.exe to PATH" from the python installation screen
  • Create any file in {FMD Directory}\lua\websitebypass and rename it to use_webdriver with no extenstion

Method 1: FlareSolverr (64-bit only)

  • Install python package requests with pip install requests
  • Download FlareSolverr. Extract it inside {FMD Directory}\lua\websitebypass
  • Make sure the .exe is in {FMD Directory}\lua\websitebypass\flaresolverr\flaresolverr.exe
  • Run flaresolverr.exe and wait for it to say Serving on http://0.0.0.0:8191
  • Paste the site url into FMD to get the manga info and chapters
  • Check Flaresolverr if any incoming requests are made and if they are solving the challenge and getting a 200 OK response

To test if everything is working properly, run in CMD or Powershell from your {FMD Directory}\lua\websitebypass directory the following command: python customcloudflare.py "https://dragontea.ink" to see if any errors pop up or not

Method 1a: FlareSolverr Fork (64-bit only)

  • If Flaresolverr has any issues solving requests make sure to check any issues and pull requests on their GitHub first
  • If there is a fork or code update without a release that fixes the issues, download the repo from GitHub by clicking <> Code in green button and click Download ZIP. The current working fork is here
  • Extract the repo clone and from inside the extracted directory, run in CMD or Powershell the following command: python src/flaresolverr.py
  • Follow the steps from method 1 from after running flaresolverr.exe

Method 2: rookiepy

  • Install python package rookiepy with pip install rookiepy
  • Open the site you want to get cookies for in any of the following browsers: Chrome, Edge, Firefox, Opera, Opera GX
  • Close the tab or browser to ensure the cookies are saved to file
  • Paste the site url into FMD to get the manga info and chapters
  • Open FMD log to see if errors pop up or not. You can ignore Please make sure FlareSolverr is running.
  • Alternatively check Options -> Websites -> Advanced and search for your site to see if cookies were applied correctly
  • If cookies were applied correctly but the manga info doesn't load, make sure you've recently visited the site in your browser and the cookies haven't expired

* Currently has issues decrypting cookies from Chrome and doesn't work. Use an alternative browser until the issue is fixed

To test if everything is working properly, run in CMD or Powershell from your {FMD Directory}\lua\websitebypass directory the following command: python cloudflare.py "https://dragontea.ink" to see if any errors pop up or not

Manual workaround
For Chromium based browser

  • Open website in browser -> press F12 -> select Network tab
  • In the Name section, select the first one -> see Headers tab and search for cookie and user-agent, copy all of it
  • In FMD, go to Options -> Websites -> Advanced -> {Website Name}, change Enabled option to True, paste the cookie and user-agent from browser
@Slasar41 Slasar41 pinned this issue Oct 10, 2024
@xavimaa
Copy link

xavimaa commented Oct 17, 2024

Hello,

Thank you for this very well-explained guide!

Is there a possibility to use flaresolverr hosted on Docker?

I apologize if this question requires opening a new issue.

Thanks again 🙏

@Slasar41
Copy link
Collaborator Author

Sorry, I don't use Docker.

@NhKPaNdA
Copy link

NhKPaNdA commented Nov 5, 2024

Is there a possibility to use flaresolverr hosted on Docker?

I tried out using a Docker and it did work in the end. I had to make sure the docker was hosting through the vm into the host machine so the cloudflare bypass could correctly connect to it through localhost.

Even though it does work I realised it does take a lot longer per connection and will generally timeout with the default 60s when connecting to the docker instead of the .exe or cmd run python which roughly takes ~20s.

@molitar
Copy link

molitar commented Feb 13, 2025

Not working.

C:\DOS\fmd 2.0\lua\websitebypass>python customcloudflare.py "https://dragontea.ink"
Error: Error solving the challenge. Timeout after 60.0 seconds.

2025-02-13 18:23:07 INFO FlareSolverr 3.3.21
2025-02-13 18:23:07 INFO Testing web browser installation...
2025-02-13 18:23:07 INFO Platform: Windows-10-10.0.19045-SP0
2025-02-13 18:23:07 INFO Chrome / Chromium path: C:\DOS\fmd 2.0\lua\websitebypass\flaresolverr\chrome\chrome.exe
2025-02-13 18:23:07 INFO Chrome / Chromium major version: 123
2025-02-13 18:23:07 INFO Launching web browser...
2025-02-13 18:23:11 INFO FlareSolverr User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36
2025-02-13 18:23:11 INFO Test successful!
2025-02-13 18:23:11 INFO Serving on http://0.0.0.0:8191
2025-02-13 18:23:25 INFO Incoming request => POST /v1 body: {'cmd': 'request.get', 'url': 'https://dragontea.ink', 'returnOnlyCookies': True, 'maxTimeout': 60000}
2025-02-13 18:23:26 INFO Challenge detected. Title found: Just a moment...
2025-02-13 18:24:29 ERROR Error: Error solving the challenge. Timeout after 60.0 seconds.
2025-02-13 18:24:29 INFO Response in 63.205 s
2025-02-13 18:24:29 INFO 127.0.0.1 POST http://localhost:8191/v1 500 Internal Server Error

Image

Image

@NhKPaNdA
Copy link

That's a known issue with flaresolver as mentioned in method 1a. Follow the fork for a working version.

@Qizmo
Copy link

Qizmo commented Feb 24, 2025

Hi, I was told to use manual workaround in the pinned issue.
#2080

So I try everything and also download chromium instead of chrome. However after I put in cookie and user-agent from browser then save, the cookie keep disappearing.

Image
First pic I put in both cookie and user-agent, then save and restart FMD2.

Image
This picture show that the cookie keep disappearing.
So Even after several trials, I still can't access the mgeko.cc What must I do?

@Qizmo
Copy link

Qizmo commented Feb 24, 2025

Next I try following: Method 1: FlareSolverr (64-bit only)
I follow everything until: Paste the site url into FMD to get the manga info and chapters

Image
I wait for it to say Serving on http://0.0.0.0:8191
Then Paste the manga site url into FMD to get the manga info and chapters as above.
But I still got nothing.

Edit: Somehow it started downloading again. ^^

@NhKPaNdA
Copy link

NhKPaNdA commented Feb 24, 2025

Hi, I was told to use manual workaround in the pinned issue. #2080

So I try everything and also download chromium instead of chrome. However after I put in cookie and user-agent from browser then save, the cookie keep disappearing.

Cookies and user agent are very tightly tied together. Make sure they're as new as possible from your browser (maybe ctrl + f5 to refresh) and probably try different mainstream browsers if one fails. Chrome, Edge, FireFox, etc.

FMD will clear the cookies and user agent if the ones that were there failed to connect which is why you're seeing them being cleared.

Next I try following: Method 1: FlareSolverr (64-bit only) I follow everything until: Paste the site url into FMD to get the manga info and chapters

Flaresolverr is not getting any requests. Try testing the python script out like in the main post:

To test if everything is working properly, run in CMD or Powershell from your {FMD Directory}\lua\websitebypass directory the following command: python customcloudflare.py "https://dragontea.ink" to see if any errors pop up or not

It's possible you're missing a python module needed to run the script.

If you get any timeouts with Flaresolverr on requests then try out method 1a.

@Qizmo
Copy link

Qizmo commented Feb 25, 2025

@NhKPaNdA IT resolved by itself already. Guess Slasar41 did something with the app.
Thank you very much ^^

@Slasar41
Copy link
Collaborator Author

@Qizmo I'm not doing anything.
The site turned off the protection so there's no need for a workaround.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants