Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

node中request模块配置项 #13

Open
raclen opened this issue Sep 15, 2019 · 0 comments
Open

node中request模块配置项 #13

raclen opened this issue Sep 15, 2019 · 0 comments
Labels
博客搬迁 以前博客的文章迁移过来

Comments

@raclen
Copy link
Owner

raclen commented Sep 15, 2019

var options = {
    url: '',
    encoding: "utf-8",
    headers: {
        "Connection": "keep-alive",
        "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
       // 'Cookie':'userid=123456; mycookie2=abcdefg',
        "Accept-Encoding": "gzip, deflate, sdch",
        "Accept-Language": "zh-CN,zh;q=0.8",
        "User-Agent": "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2603.0 Safari/537.36"
    },
    json: true
};

设置method
options.method= 'GET'
设置referer
options.headers.Referer=referer;
request默认是不支持cookies,需要开启
var j = request.jar();options.jar=j
这是操作cookies的一个例子

//格式化cookies
function formatCookies(c) {
    var cookies = {};
    var pairs = c.split(/[;,] */);
    for (var i = 0; i < pairs.length; i++) {
        var idx = pairs[i].indexOf('=');
        var key = pairs[i].substr(0, idx);
        var val = pairs[i].substr(++idx, pairs[i].length).trim();
        cookies[key] = val;
    }
    return cookies;
}

app.get('/ptqrshow', function (req, res, next) {
    var j = request.jar();
    var url = "";
    var referer = "";
    options.url=url,options.jar=j,options.method= 'GET',options.headers.Referer=referer;
    request(options, function (error, response, body) {
        if (!error && response.statusCode == 200) {
            var cookie_string = j.getCookieString(url);
            //console.log(cookie_string);
            var item = {
                cookies: cookie_string,
                url: 'image/doodle.png'
            };
            res.send(item)
        }
    }).pipe(fs.createWriteStream('./public/image/doodle.png'));;
});

如果需要传参数,可以使用form,例如

options.form={r:{"ptwebqq":cookies.ptwebqq,"clientid":53999199,"psessionid":"","status":"online"}};

设置代理(这样可以使用fiddler抓取请求了)
proxy: "http://127.0.0.1:8888", //for fiddler
抓取被墙的网站:

request({url:"https://www.facebook.com/",proxy:"http://127.0.0.1:1084"}, function (error, response, body) {
    console.log("proxy="+response.statusCode)
    if (!error && response.statusCode == 200) {
        console.log(body);
    }
});//1084 is ss proxy

request文档地址

@raclen raclen added the 博客搬迁 以前博客的文章迁移过来 label Sep 15, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
博客搬迁 以前博客的文章迁移过来
Projects
None yet
Development

No branches or pull requests

1 participant