【半成品】NGINX FastCGI purge and preload for single URL

WARNING: This article may be obsolete
This post was published in 2021-09-05. Obviously, expired content is less useful to users if it has already pasted its expiration date.

This article is categorized as "Garbage" . It should NEVER be appeared in your search engine's results.

关于FastCGI的前情提要：

以及

和Nginx Helper插件不同，我不希望我的FastCGI缓存紧密跟随我修改文章的动作；我希望我能一次性完成所有修改以后在FastCGI重新载入一个新的站点。所以我写了一个小程序：

本文探讨对单个FastCGI缓存文件的计算和处理方式。

关于FastCGI在磁盘上的的缓存路径

首先假定我们设定FastCGI的配置文件包含： fastcgi_cache_path /var/run/fastcgi-cache levels=1:2 ，

我们有一个url，假设是： https://example.com/hello-world/  （注意：trailing slash存在或不存在是完全不同的结果）

那么我们把它改为： httpsGETexample.com/hello-world/ 

然后对它进行MD5处理（比如使用命令： $ echo -n 'httpsGETexample.com/hello-world/' | md5sum ），得到：

（注意配置文件要符合levels=1:2： fastcgi_cache_path /var/run/fastcgi-cache levels=1:2 ）

然后取MD5结果的最后一个字符 5 ，再取MD5结果的倒数第3和倒数第2个字符的组合 cd ，得到路径：

这个路径就是 https://example.com/hello-world/ 对应缓存的路径。

import hashlib

url = "https://example.com/hello-world/"
digest_url = url.replace("://", "GET")
filename = hashlib.md5(digest_url.encode('utf-8')).hexdigest()
path = '/var/run/fastcgi-cache/' + filename[-1] + '/' + filename[-3:-1] + '/' + filename
print(path)

参考：

https://www.digitalocean.com/community/tutorials/how-to-setup-fastcgi-caching-with-nginx-on-your-vps

原始思路草稿（已废弃）

上面写的python preload小程序一般会和 rm -rf /your/path/to/FastCGI/cache/* 命令一起使用；如果仅仅修改一篇文章的内容又要执行这个命令，还是会亏损一些时间（站点越大、服务器性能越低，亏损的时间就越多），所以我计划了一个辅助程序，思路如下：

但是目前只写了一个简单的程序，仅仅能输入一个url：

# 需要修改:example.com
# 需要修改：/var/run/path

import hashlib
import os
from pathlib import Path
import requests

from requests.packages.urllib3.exceptions import InsecureRequestWarning

requests.packages.urllib3.disable_warnings(InsecureRequestWarning)


def download(url):
    headers = {'Host': 'example.com'}
    response = requests.get(url, headers=headers, verify=False)
    response_headers = response.headers
    print('-------' + url + '-------' + response_headers.get('nginx-cache'))


# purge_url：传入一个127.0.0.1:443规则的url，删除它的缓存；如果它存在embed，就把embed加入purge_url_list，然后删除embed的缓存
# 任何url在传入之前都需要加入就把embed加入purge_url_list
def purge_url(url):
    global purge_url_list
    digest_url = url.replace("://", "GET").replace("127.0.0.1:443", "example.com")
    filename = hashlib.md5(digest_url.encode('utf-8')).hexdigest()
    path = '/var/run/path/' + filename[-1] + '/' + filename[-3:-1] + '/' + filename
    if Path(path).is_file():
        print('detect fastcgi cache:  ' + path)
        os.remove(path)
    else:
        print('---------------detect url not in fastcgi---------------' + url)

    # 判断embed
    digest_url_embed = digest_url + 'embed/'
    filename_embed = hashlib.md5(digest_url_embed.encode('utf-8')).hexdigest()
    path_embed = '/var/run/path/' + filename_embed[-1] + '/' + filename_embed[-3:-1] + '/' + filename_embed
    if Path(path_embed).is_file():
        purge_url_list.append(url + 'embed/')
        print('detect /embed/ in fastcgi cache:  ' + path_embed)
        os.remove(path_embed)


def purge_only_content(url):
    purge_url_list.append(url)
    purge_url(url)


if __name__ == '__main__':
    purge_url_list = []

    print('【只能输入一个post或page链接】你必须确保：你仅仅修改了文章的内容，没有修改文章的标题、url、category、tag、缩略图...')
    url = input()
    url = url.replace("example.com", "127.0.0.1:443")
    download(url)
    print('

')
    purge_only_content(url)
    print('

')

    for item in purge_url_list:
        download(item)

    print('

')