2019年9月

p值,也就是cookie中的P00001,每隔一定时间后会失效,通过这个接口可以得到新的p值。
curl 命令
其中cookie_string的形式是key1:value1;key2:value2

curl -e https://www.iqiyi.com -b "cookie_string" https://passport.iqiyi.com/apis/reglogin/renew_authcookie.action?ptid=01010021010000000000

正常情况下返回如下

{
    "msg": null,
    "code": "A00000",
    "data": "p值"
}

示意如图
refresh_iqiyi_p.png

该接口是在登录情况下cookie某个值过期后需要刷新
网页中的接口请求的js代码函数getLoginStatusFromInterface

getLoginStatusFromInterface() {
    var e = this;
    return Object(f["a"])((function * () {
        const t = "https://passport.iqiyi.com/apis/reglogin/renew_authcookie.action",
        o = yield e.getRenewParams(),
        r = {
            params: o,
            interceptor: {
                needValidation: !1
            },
            dataType: "jsonp"
        };
        return yield Object(b["a"])(t, r)
    }))()
}

函数getLoginStatusFromInterface调用的js代码是

initPopupMessage() {
    var t = this;
    return Object(f["a"])((function * () {
        if (!t.hasLogin) return void(t.showVipMessage = !1);
        const o = e.cookie.get("QC163");
        if (o) t.showLoginExpiredTip = !1,
        t.initVipMessage();
        else {
            const e = yield t.getLoginStatusFromInterface(),
            o = "A00001" === e.code;
            t.resetLoginTimerCookie(),
            o ? (t.userLogoutFun(), t.showLoginExpiredTip = !0) : (t.showLoginExpiredTip = !1, t.initVipMessage())
        }
    }))()
}

通过QC163顺藤摸瓜发现这一段代码

resetLoginTimerCookie() {
    e.cookie.set("QC163", "1", {
        expires: 864e5,
        domain: ".iqiyi.com"
    })
}

其中864e586400000
1 天 = 24*60*60*1000 ms
所以p值需要一天内刷新一次

当然它也可能是假4K,腾讯嘛,你懂的。
不过这个接口需要cookie,不一定要会员的(未验证)。
请求如下:

  • 其中guid和timeforhj已省略,他们分别是数字字母32位,时间戳13位,guid不确定是不是随机的。
https://tv.aiseet.atianqi.com/i-tvbin/qtv_video/cover_details/cover_detail_waterfall?format=jce&pure_child_mode=false&position_vid=k0032m2d7bn&pip_support=no&req_scene=video_list&id=o0ytzgvq6o08e9o&id_type=2003&hv=1&Q-UA=QV%3D1%26PR%3DVIDEO%26PT%3DSNMAPP%26CHID%3D13052%26RL%3D1920*1080%26VN%3D4.6.0%26VN_CODE%3D6606%26SV%3D4.4.4%26SI%3D19%26DV%3Dorange%26VN_BUILD%3D1017%26MD%3DS2001A%26BD%3Dorange%26MF%3DMStar%2BSemiconductor%252C%2BInc.%26TVKPlatform%3D670603&guid=&licence=snm&timeforhj=
cookie:cookie string ...
Host: tv.aiseet.atianqi.com
Accept-Encoding: gzip
User-Agent: okhttp/${project.version}

话说MStar Semiconductor, Inc这里什么鬼,搜了下叫晨星半导体股份有限公司,和腾讯有啥关系吗。

  • 最终可以精简如下,可能还能去掉一些参数,不过我不想弄了。请求其他的可有可无,但要判断是不是4K,必须带cookie
https://tv.aiseet.atianqi.com/i-tvbin/qtv_video/cover_details/cover_detail_waterfall?format=jce&req_scene=video_list&id=o0ytzgvq6o08e9o&Q-UA=QV%3D1%26PR%3DVIDEO%26PT%3DSNMAPP%26CHID%3D13052%26RL%3D3840*2160%26VN%3D4.6.0%26VN_CODE%3D6606%26SV%3D%26SI%3D%26DV%3D%26VN_BUILD%3D%26MD%3D%26BD%3D%26MF%3D%26TVKPlatform%3D670603
  • Q-UA部分
QV%3D1%26PR%3DVIDEO%26PT%3DSNMAPP%26CHID%3D13052%26RL%3D3840*2160%26VN%3D4.6.0%26VN_CODE%3D6606%26SV%3D%26SI%3D%26DV%3D%26VN_BUILD%3D%26MD%3D%26BD%3D%26MF%3D%26TVKPlatform%3D670603

urldecode后,其中3840*2160可以没有,其他部分似乎都是必须的

QV=1&PR=VIDEO&PT=SNMAPP&CHID=13052&RL=3840*2160&VN=4.6.0&VN_CODE=6606&SV=&SI=&DV=&VN_BUILD=&MD=&BD=&MF=&TVKPlatform=670603

urlencode代码(python)

from urllib.parse import quote
qua_d = "QV=1&PR=VIDEO&PT=SNMAPP&CHID=13052&RL=3840*2160&VN=4.6.0&VN_CODE=6606&SV=&SI=&DV=&VN_BUILD=&MD=&BD=&MF=&TVKPlatform=670603"
qua_e = quote(qua_d,safe='*')
# QV%3D1%26PR%3DVIDEO%26PT%3DSNMAPP%26CHID%3D13052%26RL%3D3840*2160%26VN%3D4.6.0%26VN_CODE%3D6606%26SV%3D%26SI%3D%26DV%3D%26VN_BUILD%3D%26MD%3D%26BD%3D%26MF%3D%26TVKPlatform%3D670603

正题

这个接口是看专辑的,并非单集,单集需要通过视频接口信息得到。
链接可直接在登录了腾讯视频的浏览器中访问,会得到一个文件。
返回数据中,如果有一个链接是http://vmat.gtimg.com/kt/common/video/imgtag/4k1514540045598.png,那么该剧集有4K(不一定是真4K,也可能是1080 hevc)
返回示例
qq_video_4k_check.png

---2019/09/13---

为了上pyd和用pyinstaller打包趟了些坑,过两天详细补上,然后弄个打包脚本(嗯一定)...

---2019/09/15---

先上个代码吧...

import os
import sys
import shutil
from time import time
from py_compile import compile as pyc_compile
from distutils.core import setup, Extension
from Cython.Distutils import build_ext

# 编译pyd时需要额外参数 放在这里 执行命令时就不用加额外参数了
sys.argv += ['build_ext', '--inplace']
project_name = "your_project_name"
spec_file_name = project_name + ".spec"

def compile_py_to(py_path, compile_type="pyd", pkgs=None, compile_name=None, compile_output_dir=None, export_func_name=None):
    # py_path py文件完整路径
    # compile_type py编译的类型 默认是输出pyd pyc可选 pyc下可以另行配置要不要保留注释 说明 文档
    # compile_name py编译最终的文件名 不含后缀 后缀由compile_type确定 不指定则默认和文件名一样
    # compile_output_dir 编译的最终输出目录 若不指定则输出目录和py同一位置
    # export_func_name 对外的接口函数名称 pyd可能需要 不指定则默认和文件名一样

    # py_dir py文件所在文件夹
    # py_name py文件名 带后缀
    # py_basename py文件名 不带后缀
    # ext 文件后缀 万一传入的不是py文件 用来判断一下
    py_dir, py_name = os.path.split(py_path)
    py_basename, ext = os.path.splitext(py_name)
    pyini = [py_path, py_dir, py_name, py_basename]
    if ext != ".py":
        print("{}\n{} 不是python文件".format(py_dir, py_name))
        return False

    if compile_type == "pyd":
        compile_status = compile_py_to_pyd(pyini, pkgs=pkgs, compile_name=compile_name, compile_output_dir=compile_output_dir, export_func_name=export_func_name)
    elif compile_type == "pyc":
        compile_status = compile_py_to_pyc(pyini, compile_name=compile_name, compile_output_dir=compile_output_dir)
    else:
        compile_status = "未作任何处理"
    print("{}处理为{}的结果是 --> {}".format(py_path, compile_type, compile_status))

def compile_py_to_pyd(pyini, pkgs=None, compile_name=None, compile_output_dir=None, export_func_name=None):
    py_path, py_dir, py_name, py_basename = pyini
    compile_c_name = py_basename + ".c" # 临时生成的c文件名
    compile_c_path = os.path.join(py_dir, compile_c_name) # 完整c文件的路径
    pyd_ends = ".cp35-win32" + ".pyd" # 前面部分由python版本和windows位数确定

    # 如果没有指定最终输出名则使用py_basename作为最终输出文件名
    # 编译的最终输出目录
    if compile_name is None:
        compile_name = py_basename
    if compile_output_dir is None:
        compile_output_dir = py_dir

    # export_func_name 如果不指定则和py文件名不带后缀一致
    if export_func_name is None:
        export_func_name = py_basename

    compile_pyd_name = export_func_name + pyd_ends # 生成的pyd文件名 注意这里用的是export_func_name
    compile_pyd_name_needed = compile_name + ".pyd" # 最终需要的pyd文件名
    compile_pyd_path = os.path.join(py_dir, compile_pyd_name) # 完整pyd文件的路径
    compile_pyd_path_needed = os.path.join(compile_output_dir, compile_pyd_name_needed) # 完整pyd文件的路径

    # 正常情况这里需要注释掉 不注释掉则将跳过已存在的pyd
    # if os.path.isfile(compile_pyd_path_needed):
    #     return True

    if pkgs:
        export_func_name = ".".join(pkgs + [export_func_name])
    # ext_modules 中Extension第二个参数实际上可以多个文件一起
    # 但暂时采用单个文件依次处理的方式 熟悉了可以试试更多的方法
    ext_modules = [
        Extension(export_func_name, [py_path]),
    ]

    # 注意 生成pyd文件命令
    # python compile_to_pyc_or_pyd.py build_ext --inplace
    setup(
        name = py_basename,# 包名
        version = "0.3.2",# 包版本
        author = "weimo",# 作者
        author_email = "[email protected]",# 作者邮件
        url = "https://blog.weimo.info",# 包的链接
        description = "build pyd file by {} file.".format(py_name), # 简单描述
        # long_description = "", # 详细描述
        cmdclass = {'build_ext': build_ext},
        ext_modules = ext_modules
    )

    # 删除临时c文件并重命名pyd文件名
    if os.path.isfile(compile_c_path):
        os.remove(compile_c_path)
    if os.path.isfile(compile_pyd_path):
        # 输出目录已存在pyd文件 先删除再重命名
        if os.path.isfile(compile_pyd_path_needed):
            os.remove(compile_pyd_path_needed)
    os.rename(compile_pyd_path, compile_pyd_path_needed)
    # 至此 默认情况下 py文件编译为pyd文件 两者位于同一目录位置
    return True

def compile_py_to_pyc(pyini, compile_name=None, compile_output_dir=None):
    py_path, py_dir, py_name, py_basename = pyini

    if compile_name is None:
        compile_name = py_basename
    if compile_output_dir is None:
        compile_output_dir = py_dir

    compile_pyc_name = compile_name + ".pyc"
    compile_pyc_path = os.path.join(compile_output_dir, compile_pyc_name)

    # 预先删除掉原来的pyc文件
    if os.path.exists(compile_pyc_path):
        os.remove(compile_pyc_path)
        print("删除了已存在的pyc文件 --> {}".format(compile_pyc_path))

    # optimize 1是去除注释 2是去除注释和文档
    compile_result = pyc_compile(py_path, cfile=compile_pyc_path, optimize=2)
    print("compile_result --> {}".format(compile_result))
    print("编译了 {} 为 {}".format(py_path, compile_pyc_path))
    return True

def console_structure(project_dir=os.getcwd()):
    # project_dir 项目目录
    # 遍历project_dir 输出所有可能需要编译的文件路径等
    # 手动选择调整 然后进行编译打包一把梭

    # 一些通用的需要排除的通用设定
    # methods = "exclude"
    flist = []
    methods = "include"
    include_ends = [".py"]
    exclude_starts = [".", "__"]
    exclude_ends = [".log", ".json", ".exe", ".txt", ".ico", ".ini", ".html", ".h", ".ui", ".c", ".cpp", ".spec"]
    exclude_dir = ["build", "dist", "program", "testscript"]
    exclude_file = ["__init__.py", "py_file_you_want_to_exclude"]
    tmp_exclude_file = ["tmp_py_file_you_want_to_exclude"]
    exclude_file += tmp_exclude_file
    for root, dirs, files in os.walk(project_dir, topdown=True):
        if root == project_dir:
            for subdir in dirs:
                needed_continue = False
                for starts in exclude_starts:
                    if subdir.startswith(starts):
                        needed_continue = True
                        break
                if needed_continue:
                    continue
                if subdir in exclude_dir:
                    continue
                flist += console_structure(project_dir=os.path.join(project_dir, subdir))
            for file in files:
                if methods == "exclude":
                    needed_continue = False
                    for ends in exclude_ends:
                        if file.endswith(ends):
                            needed_continue = True
                            break
                    if needed_continue:
                        continue
                if file in exclude_file:
                    continue
                if file.endswith(".py"):
                    py_path = os.path.join(project_dir, file)
                    print("r'" + py_path + "',")
                    flist.append(py_path)
        else:
            break
        return flist

def copy_to_zip(project_tmp_compile_dir):
    mv_exclude_files = ["some_files_dont_need_to_move_such_as_main_pyd", spec_file_name]
    mv_exclude_folders = ["some_folders_dont_need_to_move_such_as_build_and_dist", "dist", "__pycache__"]
    for root, dirs, files in os.walk(project_tmp_compile_dir, topdown=True):
        if root == project_tmp_compile_dir:
            for subdir in dirs:
                if subdir in mv_exclude_folders:
                    continue
                path_old = os.path.join(project_tmp_compile_dir, subdir)
                path_new = os.path.join(project_tmp_compile_dir, "dist", project_name)
                shutil.move(path_old, path_new)
            for file in files:
                if file in mv_exclude_files:
                    continue
                path_old = os.path.join(project_tmp_compile_dir, file)
                path_new = os.path.join(project_tmp_compile_dir, "dist", project_name)
                shutil.move(path_old, path_new)

def copy_binaries(project_tmp_compile_dir, project_dir=os.getcwd()):
    dirs_need_to_copy = ["icon", "program"]
    files_need_to_copy = [spec_file_name, "file_need_to_copy"]
    for folder_need_copy in dirs_need_to_copy:
        path_old = os.path.join(project_dir, folder_need_copy)
        path_new = os.path.join(project_tmp_compile_dir, folder_need_copy)
        if not os.path.exists(path_new):
            shutil.copytree(path_old, path_new)
    for file_need_copy in files_need_to_copy:
        path_old = os.path.join(project_dir, file_need_copy)
        path_new = os.path.join(project_tmp_compile_dir, file_need_copy)
        if os.path.exists(path_new):
            # pyinstaller 打包用的spec文件 和 另一个py文件需要覆盖掉
            os.remove(path_new)
        shutil.copyfile(path_old, path_new)

def compile_it(flist, project_dir=os.getcwd()):
    project_top_level_dir, project_name = os.path.split(project_dir)
    project_tmp_compile_dir = os.path.join(project_top_level_dir, "tmp_compile_for_" + project_name)
    compile_config = {
        "project_name":project_name,
        "project_version":"0.3.2",
        "project_tmp_compile_dir":project_tmp_compile_dir,
    }
    if not os.path.isdir(project_tmp_compile_dir):
        os.mkdir(project_tmp_compile_dir)
    pycs = [os.path.join(project_dir, "compact.py")]
    check_path_to_make_dir_and_compile(pycs, project_dir, project_tmp_compile_dir)
    pyds = flist
    check_path_to_make_dir_and_compile(pyds, project_dir, project_tmp_compile_dir, compile_type="pyd")
    return project_tmp_compile_dir

def check_path_to_make_dir_and_compile(pycs_or_pyds, project_dir, project_tmp_compile_dir, compile_type="pyc"):
    for py__file_path in pycs_or_pyds:
        # 先得到相对于项目的相对路径
        # 然后分割 得到可能需要新建的文件夹 需要则新建 不需要则开始编译
        pyc_file_relpath = os.path.relpath(py__file_path, project_dir)
        pyc_file_path_pres = os.path.normpath(pyc_file_relpath)
        dirs_need_to_make = pyc_file_path_pres.split(os.sep)[:-1] # 注意最后一个是文件 所以要去掉
        # 检查有没有已经新建 没有的话就新建
        base_dir = project_tmp_compile_dir
        for dir_make in dirs_need_to_make:
            dir_make_full_path = os.path.join(base_dir, dir_make)
            if os.path.isdir(dir_make_full_path):
                base_dir = dir_make_full_path # 注意这里是因为往后的话 基础目录在变化
                continue
            else:
                # print("新建 --> {}".format(dir_make_full_path))
                os.mkdir(dir_make_full_path)
        compile_output_dir = os.path.join(project_tmp_compile_dir, *dirs_need_to_make)
        if compile_type == "pyc":
            compile_py_to(py__file_path, compile_type="pyc", compile_output_dir=compile_output_dir)
        elif compile_type == "pyd":
            py__dir, py_name = os.path.split(py__file_path)
            py__basename, ext = os.path.splitext(py_name)
            if py__basename == "main":
                compile_py_to(py__file_path, compile_type="pyd", pkgs=dirs_need_to_make, compile_name="compile_name_you_want", compile_output_dir=compile_output_dir, export_func_name="export_func_name_you_want")
            else:
                compile_py_to(py__file_path, compile_type="pyd", pkgs=dirs_need_to_make, compile_output_dir=compile_output_dir)

def pyinstaller_build_tozip(project_tmp_compile_dir):
    # 这里实际上没有打包成zip 因为还需要测试下 稳定了加上
    copy_binaries(project_tmp_compile_dir)
    path_log = os.getcwd()
    os.chdir(project_tmp_compile_dir)
    os.system("pyinstaller {}".format(spec_file_name))
    os.chdir(path_log)
    copy_to_zip(project_tmp_compile_dir)

if __name__ == "__main__":
    flist = console_structure()
    print("-*-" * 10)
    compile_flag = input("continue to compile? (y/n)\n")
    if compile_flag != "y":
        print("action end.")
        exit()
    ts = time()
    project_tmp_compile_dir = compile_it(flist)
    pyinstaller_build_tozip(project_tmp_compile_dir)
    print("打包完成!耗时{:.2f}s".format(time() - ts))

  • typecho_metas下面存有分类名字,首先通过这张表选出mid
  • 然后从typecho_relationships中选出属于该分类的全部cid,即文章id
  • 更新typecho_contents中的status即可改变文章公开度
USE typecho;
UPDATE typecho_contents SET STATUS = "hidden"
WHERE cid IN (
SELECT cid
FROM typecho_relationships
WHERE MID = (
SELECT MID
FROM typecho_metas
WHERE NAME = "分类名字"));

前一篇说到很多乱七八糟的访问记录,其中SemrushBot最多,UA也是怪怪的,于是禁止之(
不过好像是含有这个字段的都会被禁止,某个特定UA没仔细看,需要的时候再说hhh
在nginx.conf同一个目录新建一个UA_deny.conf文件
文件内容如下

if ($http_user_agent ~ "SemrushBot"){
     return 403;
}

然后再特定的server{}中添加一句

include UA_deny.conf;

用curl测试返回结果如下

curl -A "Mozilla/5.0 (compatible; SemrushBot/6~bl; +http://www.semrush.com/bot.html)" https://blog.weimo.info
<html>
<head><title>403 Forbidden</title></head>
<body bgcolor="white">
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.6.2</center>
</body>
</html>

由于网站加了cdn,禁止ip需要禁的是用户真实ip而不是cdn的ip,所以得做一些处理。
在nginx.conf中加入下面的部分

map $HTTP_CF_CONNECTING_IP $clientRealIp 
{
    "" $remote_addr;
    ~^(?P<firstAddr>[a-z0-9.:]+),?.*$ $firstAddr;
}

然后还是里面,当然也可以另外新建一个文件,这里我就偷个懒了。。(嗯,没错,5.188.84.*这个段一直发垃圾评论)

if ($clientRealIp ~ "5.188.84."){
     return 403;
}