【20190724】Prerender nginx 配置

[zhongxia245]

适用 eggjs + puppeteer 实现了一个抓取 CSR 页面的内容。

完成了服务，应该在 nginx 配置代理，把爬虫代理到SEO 服务来， 下面便是接口重写的 nginx 配置

location / {
    root   /usr/local/etc/nginx/html;
    index  index.html index.htm;

    set $prerender 0;
    if ($http_user_agent ~* "googlebot|bingbot|yandex|baiduspider|twitterbot|facebookexternalhit|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest\/0\.|pinterestbot|slackbot|vkShare|W3C_Validator") {
        set $prerender 1;
    }
    if ($args ~ "_escaped_fragment_") {
        set $prerender 1;
    }
    if ($http_user_agent ~ "Prerender") {
        set $prerender 0;
    }
    if ($uri ~* "\.(js|css|xml|less|png|jpg|jpeg|gif|pdf|doc|txt|ico|rss|zip|mp3|rar|exe|wmv|doc|avi|ppt|mpg|mpeg|tif|wav|mov|psd|ai|xls|mp4|m4a|swf|dat|dmg|iso|flv|m4v|torrent|ttf|woff|svg|eot)") {
        set $prerender 0;
    }

    if ($prerender = 1) {
       # rewrite ^/(.*) http://www.aaaa.com/html?url=$1;
        rewrite ^/(.*) http://your_domain.com/your_path?url=$1;
    }
}