帶你用python寫一個下載文件的小工具

Python

關於Python，相信很多人都有所耳聞。經過近三十年的發展，再加上近幾年AI的興起，Python作為一種科學語言的流行程度急劇上升。而Python語法簡單、語句清晰的特點，可以讓程序員把精力集中在業務邏輯上，減輕我們的壓力。畢竟人生苦短，我用Python。看看Python之父--Guido van Rossum的髮量，在對比下Java之父——James Gosling。。。

Python之父

Java之父

開發環境

• Python3.6.5
• pip(Python包管理工具，Python3.4以上版本都會自帶)
• 一個順手的編輯器（像這種一百多行代碼的小工具，隨便一個編輯器都可以，推薦vsCode，畢竟微軟大法好）

需求分析

我們需要做一個下載文件並壓縮保存的小工具，最好能多個目的地址同時下載。所以我們會用到以下功能：

• ftp（引入paramiko包，Python支持ftp的包）
• threading（Python對多線程支持的包）
• 配置文件（需要引入json包，來解析json格式的配置文件，json格式簡單明瞭而且解析方便，所以選擇json格式配置文件）
• os(要操作目錄)
• 需要記錄日誌（logging，Python日誌相關的包）
• zipfile（Python壓縮zip文件需要用到的包）

開始擼代碼

引入必要的包

<code>import paramiko
import threading
import json
import os
import sys from stat 
import S_ISDIR as isdir
import logging
import zipfile
import time/<code>

獲取ftp連接

<code>#建立連接，獲取sftp句柄

def sftp_connect(username,password,host,port):

    global logger

    client = None

    sftp = None

    try:

        client = paramiko.Transport((host,port))

    except Exception as error:

        logger.error(error)

    else:

        try:

            client.connect(username=username, password=password)

        except Exception as error:

            logger.error(error)

        else:

            sftp = paramiko.SFTPClient.from_transport(client)

    return client,sftp/<code>

斷開ftp連接

<code>#斷開連接

def disconnect(client):

    global logger

    try:

        client.close()
 

    except Exception as error:

        logger.error(error)



def check_local(localPath):

    global logger

    if not os.path.exists(localPath):

        try:

            os.mkdir(localPath)

        except IOError as err:

            logger.error(err)/<code>

從ftp服務器下載文件

<code>#下載方法

def downLoad(client, sftp, remote, local):

    global logger

    #檢查遠程文件

    try:

        result = sftp.stat(remote)

    except IOError as err:

        error = '[ERROR %s] %s: %s' %(err.errno,os.path.basename(os.path.normpath(remote)),err.strerror)

        logger.error(error)

    else:

        if isdir(result.st_mode):

            dirname = os.path.basename(os.path.normpath(remote))

            local = os.path.join(local, dirname)

            #local = local.replace("\\\","/") 


            check_local(local)

            for file in sftp.listdir(remote):

                sub_remote = os.path.join(remote, file)

                sub_remote = sub_remote.replace("\\\","/")

                downLoad(client, sftp,sub_remote,local)

        else:

            if os.path.isdir(local):

                local = os.path.join(local, os.path.basename(remote))

            try:

                sftp.get(remote, local)

            except IOError as err:

                logger.error(err)

            else:

                logger.info('[get] %s %s %s', remote, '==>', local)

                lock.acquire()

                global finish

                finish += 1

                lock.release()

                logger.info('已下載 [%d] 個文件', finish)/<code>

壓縮文件

<code>#壓縮文件

def writeAllFileToZip(absDir,zipFile):

    global logger

    for f in os.listdir(absDir):
 

        absFile=os.path.join(absDir,f) #子文件的絕對路徑

        if os.path.isdir(absFile): #判斷是文件夾，繼續深度讀取。

            zipFile.write(absFile) #在zip文件中創建文件夾

            logger.info('寫入 %s 到壓縮包 %s 成功', absFile, zipFile.filename)

            writeAllFileToZip(absFile,zipFile) #遞歸操作

        else: #判斷是普通文件，直接寫到zip文件中。

            zipFile.write(absFile)

            logger.info('寫入 %s 到壓縮包 %s 成功', absFile, zipFile.filename)

    return/<code>

主方法（程序入口）

<code>if __name__ == "__main__":

    #日誌

    logging.basicConfig(level=logging.INFO,format ='%(asctime)s - %(name)s - %(levelname)s - %(message)s')

    logger = logging.getLogger()

    logFormatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')

    handler = logging.FileHandler("log.txt")

    handler.setLevel(logging.INFO)

    handler.setFormatter(logFormatter)

    console = logging.StreamHandler()

    console.setLevel(logging.INFO)

    console.setFormatter(logFormatter)

    logger.addHandler(handler) 


    #logger.addHandler(console)

    #鎖

    lock = threading.Lock()

    #條件

    finish = 0

    #讀取配置文件

    configFile = open('config.json')

    configs = json.loads(configFile.read())

    configFile.close()



    logger.info('工作目錄：%s', configs['workDir'])

    #檢查本地工作目錄（父目錄）

    check_local(configs['workDir'])

    threadList = []

    for config in configs['threads']:

        logger.info("=======================配置信息 start=============================")

        logger.info("HostAddress %s",config['HostAddress'])

        logger.info('Port %s', config['Port'])

        logger.info('Username %s',config['Username'])

        logger.info('Password %s',config['Password'])

        logger.info('RemotePath %s',config['RemotePath'])

        logger.info('LocalPath %s',config['LocalPath'])

        logger.info("=======================配置信息 end=============================")
 

        client,sftp = sftp_connect(config['Username'],config['Password'],config['HostAddress'],config['Port'])

        #創建本地目錄

        check_local(config['LocalPath'])

        #多線程

        t = threading.Thread(target=downLoad, args=(client, sftp, config['RemotePath'], config['LocalPath']))

        t.start()

        threadList.append(t)

        #單線程

        #downLoad(client, sftp, config['RemotePath'], config['LocalPath'])

    for t in threadList:

        t.join()



    #壓縮

    # while threading.active_count() == 1:

    logger.info('=======================準備開始壓縮==========================')

    #time.sleep(3)

    logger.info('==========================開始壓縮。。。==========================')

    check_local(configs['zipDir'])

    zipFilePath = os.path.join(configs['zipDir'], configs['zipName'] + '.zip')

    zipFile = zipfile.ZipFile(zipFilePath, 'w', zipfile.ZIP_DEFLATED, allowZip64=True)

    absZipFileDir = configs['workDir']

    writeAllFileToZip(absZipFileDir, zipFile)

    logger.info('==========================壓縮完成==========================')

    #保留終端 


    print("Press Enter to continue ...")

    input()/<code>

配置文件

<code>{

    "workDir": "F:/temp",

    "zipDir": "F:/result",

    "zipName": "rrrr",

    "threads": [{

            "HostAddress": "10.7.5.69",

            "Port": 22,

            "Username": "ftpuser",

            "Password": "ftpuser",

            "RemotePath": "/app/ftpuser/100_OrdMember_2019",

            "LocalPath": "F:/temp/1"

        },

        {

            "HostAddress": "10.7.5.69",

            "Port": 22,

            "Username": "ftpuser",

            "Password": "ftpuser",

            "RemotePath": "/app/ftpuser/100_OrdMember_2019",

            "LocalPath": "F:/temp/2"

        },

        {

            "HostAddress": "10.7.5.69", 


            "Port": 22,

            "Username": "ftpuser",

            "Password": "ftpuser",

            "RemotePath": "/app/ftpuser/100_OrdMember_2019",

            "LocalPath": "F:/temp/3"

        }

    ]

}/<code>

源碼講解

這裡我們做的其實是一個Python腳本，像是shell腳本。程序入口在主方法，進入主方法後我們幹了幾件事：

聲明日誌，固定了日誌輸出的格式、日誌等級和輸出日誌的目的地（日誌文件和終端）。輸出日誌到日誌文件用於保存日誌，便於日後查看，輸出到終端是讓用戶及時瞭解程序在做什麼，做到什麼程度了。

聲明一個鎖。這個所是為了下載文件計數，保證下載文件時，只有一個線程可以操作變量finish。

讀取配置文件。想要開始執行，得先獲取咱們的配置文件，讓程序知道它連接那一臺服務器，下載哪個目錄下的文件到本地哪個目錄，等等……

檢查本地工作目錄。為了減少用戶配置的錯誤，避免他/她配了一個不存在的地址導致程序找不到目錄而引發錯誤，所以我們要判斷目錄是否存在，是否需要創建目錄。

根據配置文件threads中的配置，循環創建線程，完成讀取本線程需要的配置文件、連接ftp、下載文件的操作。

壓縮文件。拼接文件目錄，保證壓縮文件的目錄關係不變，調用壓縮文件方法。

美化結束動作，如果沒有這一步，程序跑完就會關閉當前終端，導致用戶無法瞭解程序是跑完了，還是中間出了問題關閉了。輸出一段文字告訴用戶可以按Enter來結束程序，並且用input()監聽用戶的輸入，達到保留終端的效果。

查看日誌文件

<code>2020-01-19 13:35:33,253 - root - INFO - 工作目錄： F:/temp
2020-01-19 13:35:33,254 - root - INFO - =======================配置信息 start=============================
2020-01-19 13:35:33,256 - root - INFO - HostAddress 10.7.5.69
2020-01-19 13:35:33,257 - root - INFO - Port 22
2020-01-19 13:35:33,258 - root - INFO - Username ftpuser
2020-01-19 13:35:33,259 - root - INFO - Password ftpuser
2020-01-19 13:35:33,260 - root - INFO - RemotePath /app/ftpuser/100_OrdMember_2019
2020-01-19 13:35:33,260 - root - INFO - LocalPath F:/temp/1
2020-01-19 13:35:33,261 - root - INFO - =======================配置信息 end=============================
2020-01-19 13:35:33,290 - paramiko.transport - INFO - Connected (version 2.0, client OpenSSH_5.5)
2020-01-19 13:35:33,638 - paramiko.transport - INFO - Authentication (password) successful!
2020-01-19 13:35:33,656 - paramiko.transport.sftp - INFO - [chan 0] Opened sftp connection (server version 3)
2020-01-19 13:35:33,659 - root - INFO - =======================配置信息 start============================= 

2020-01-19 13:35:33,660 - root - INFO - HostAddress 10.7.5.69
2020-01-19 13:35:33,661 - root - INFO - Port 22
2020-01-19 13:35:33,661 - root - INFO - Username ftpuser
2020-01-19 13:35:33,662 - root - INFO - Password ftpuser
2020-01-19 13:35:33,663 - root - INFO - RemotePath /app/ftpuser/100_OrdMember_2019
2020-01-19 13:35:33,664 - root - INFO - LocalPath F:/temp/2
2020-01-19 13:35:33,665 - root - INFO - =======================配置信息 end=============================
2020-01-19 13:35:33,680 - paramiko.transport - INFO - Connected (version 2.0, client OpenSSH_5.5)
2020-01-19 13:35:33,987 - root - INFO - [get] /app/ftpuser/100_OrdMember_2019/100_OrdMember_2019/100_OrdMember_2019222 ==> F:/temp/1\\100_OrdMember_2019\\100_OrdMember_2019\\100_OrdMember_2019222
2020-01-19 13:35:33,987 - root - INFO - 已下載 [1] 個文件
2020-01-19 13:35:34,011 - root - INFO - [get] /app/ftpuser/100_OrdMember_2019/100_OrdMember_2019/100_OrdMember_201922 ==> F:/temp/1\\100_OrdMember_2019\\100_OrdMember_2019\\100_OrdMember_201922
2020-01-19 13:35:34,012 - root - INFO - 已下載 [2] 個文件
2020-01-19 13:35:34,032 - paramiko.transport - INFO - Authentication (password) successful!
2020-01-19 13:35:34,047 - paramiko.transport.sftp - INFO - [chan 0] Opened sftp connection (server version 3)
2020-01-19 13:35:34,050 - root - INFO - =======================配置信息 start=============================
2020-01-19 13:35:34,051 - root - INFO - HostAddress 10.7.5.69
2020-01-19 13:35:34,052 - root - INFO - Port 22
2020-01-19 13:35:34,052 - root - INFO - Username ftpuser
2020-01-19 13:35:34,053 - root - INFO - Password ftpuser
2020-01-19 13:35:34,054 - root - INFO - RemotePath /app/ftpuser/100_OrdMember_2019
2020-01-19 13:35:34,055 - root - INFO - LocalPath F:/temp/3
2020-01-19 13:35:34,056 - root - INFO - =======================配置信息 end=============================
2020-01-19 13:35:34,060 - root - INFO - [get] /app/ftpuser/100_OrdMember_2019/tmp001.xls ==> F:/temp/1\\100_OrdMember_2019\\tmp001.xls
2020-01-19 13:35:34,071 - root - INFO - 已下載 [3] 個文件
2020-01-19 13:35:34,072 - paramiko.transport - INFO - Connected (version 2.0, client OpenSSH_5.5)
2020-01-19 13:35:34,095 - root - INFO - [get] /app/ftpuser/100_OrdMember_2019/100_OrdMember_2019111 ==> F:/temp/1\\100_OrdMember_2019\\100_OrdMember_2019111
2020-01-19 13:35:34,096 - root - INFO - 已下載 [4] 個文件
2020-01-19 13:35:34,118 - root - INFO - [get] /app/ftpuser/100_OrdMember_2019/100_OrdMember_2019/100_OrdMember_2019222 ==> F:/temp/2\\100_OrdMember_2019\\100_OrdMember_2019\\100_OrdMember_2019222
2020-01-19 13:35:34,118 - root - INFO - 已下載 [5] 個文件
2020-01-19 13:35:34,123 - root - INFO - [get] /app/ftpuser/100_OrdMember_2019/100_OrdMember_201911 ==> F:/temp/1\\100_OrdMember_2019\\100_OrdMember_201911
2020-01-19 13:35:34,124 - root - INFO - 已下載 [6] 個文件
2020-01-19 13:35:34,258 - root - INFO - [get] /app/ftpuser/100_OrdMember_2019/100_OrdMember_2019/100_OrdMember_201922 ==> F:/temp/2\\100_OrdMember_2019\\100_OrdMember_2019\\100_OrdMember_201922
2020-01-19 13:35:34,258 - root - INFO - 已下載 [7] 個文件
2020-01-19 13:35:34,449 - root - INFO - [get] /app/ftpuser/100_OrdMember_2019/tmp001.xls ==> F:/temp/2\\100_OrdMember_2019\\tmp001.xls
2020-01-19 13:35:34,450 - root - INFO - 已下載 [8] 個文件
2020-01-19 13:35:34,464 - paramiko.transport - INFO - Authentication (password) successful!
2020-01-19 13:35:34,479 - root - INFO - [get] /app/ftpuser/100_OrdMember_2019/100_OrdMember_2019111 ==> F:/temp/2\\100_OrdMember_2019\\100_OrdMember_2019111
2020-01-19 13:35:34,480 - root - INFO - 已下載 [9] 個文件
2020-01-19 13:35:34,480 - paramiko.transport.sftp - INFO - [chan 0] Opened sftp connection (server version 3)
2020-01-19 13:35:34,508 - root - INFO - [get] /app/ftpuser/100_OrdMember_2019/100_OrdMember_201911 ==> F:/temp/2\\100_OrdMember_2019\\100_OrdMember_201911
2020-01-19 13:35:34,509 - root - INFO - 已下載 [10] 個文件
2020-01-19 13:35:34,552 - root - INFO - [get] /app/ftpuser/100_OrdMember_2019/100_OrdMember_2019/100_OrdMember_2019222 ==> F:/temp/3\\100_OrdMember_2019\\100_OrdMember_2019\\100_OrdMember_2019222
2020-01-19 13:35:34,552 - root - INFO - 已下載 [11] 個文件 

2020-01-19 13:35:34,576 - root - INFO - [get] /app/ftpuser/100_OrdMember_2019/100_OrdMember_2019/100_OrdMember_201922 ==> F:/temp/3\\100_OrdMember_2019\\100_OrdMember_2019\\100_OrdMember_201922
2020-01-19 13:35:34,576 - root - INFO - 已下載 [12] 個文件
2020-01-19 13:35:34,624 - root - INFO - [get] /app/ftpuser/100_OrdMember_2019/tmp001.xls ==> F:/temp/3\\100_OrdMember_2019\\tmp001.xls
2020-01-19 13:35:34,624 - root - INFO - 已下載 [13] 個文件
2020-01-19 13:35:34,649 - root - INFO - [get] /app/ftpuser/100_OrdMember_2019/100_OrdMember_2019111 ==> F:/temp/3\\100_OrdMember_2019\\100_OrdMember_2019111
2020-01-19 13:35:34,649 - root - INFO - 已下載 [14] 個文件
2020-01-19 13:35:34,675 - root - INFO - [get] /app/ftpuser/100_OrdMember_2019/100_OrdMember_201911 ==> F:/temp/3\\100_OrdMember_2019\\100_OrdMember_201911
2020-01-19 13:35:34,675 - root - INFO - 已下載 [15] 個文件
2020-01-19 13:35:34,677 - root - INFO - =======================準備開始壓縮==========================
2020-01-19 13:35:34,678 - root - INFO - ==========================開始壓縮。。。==========================
2020-01-19 13:35:34,682 - root - INFO - 寫入 F:/temp\\1 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,684 - root - INFO - 寫入 F:/temp\\1\\100_OrdMember_2019 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,686 - root - INFO - 寫入 F:/temp\\1\\100_OrdMember_2019\\100_OrdMember_2019 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,696 - root - INFO - 寫入 F:/temp\\1\\100_OrdMember_2019\\100_OrdMember_2019\\100_OrdMember_201922 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,698 - root - INFO - 寫入 F:/temp\\1\\100_OrdMember_2019\\100_OrdMember_2019\\100_OrdMember_2019222 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,700 - root - INFO - 寫入 F:/temp\\1\\100_OrdMember_2019\\100_OrdMember_201911 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,702 - root - INFO - 寫入 F:/temp\\1\\100_OrdMember_2019\\100_OrdMember_2019111 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,716 - root - INFO - 寫入 F:/temp\\1\\100_OrdMember_2019\\tmp001.xls 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,717 - root - INFO - 寫入 F:/temp\\2 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,718 - root - INFO - 寫入 F:/temp\\2\\100_OrdMember_2019 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,720 - root - INFO - 寫入 F:/temp\\2\\100_OrdMember_2019\\100_OrdMember_2019 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,723 - root - INFO - 寫入 F:/temp\\2\\100_OrdMember_2019\\100_OrdMember_2019\\100_OrdMember_201922 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,738 - root - INFO - 寫入 F:/temp\\2\\100_OrdMember_2019\\100_OrdMember_2019\\100_OrdMember_2019222 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,740 - root - INFO - 寫入 F:/temp\\2\\100_OrdMember_2019\\100_OrdMember_201911 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,742 - root - INFO - 寫入 F:/temp\\2\\100_OrdMember_2019\\100_OrdMember_2019111 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,745 - root - INFO - 寫入 F:/temp\\2\\100_OrdMember_2019\\tmp001.xls 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,768 - root - INFO - 寫入 F:/temp\\3 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,770 - root - INFO - 寫入 F:/temp\\3\\100_OrdMember_2019 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,772 - root - INFO - 寫入 F:/temp\\3\\100_OrdMember_2019\\100_OrdMember_2019 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,774 - root - INFO - 寫入 F:/temp\\3\\100_OrdMember_2019\\100_OrdMember_2019\\100_OrdMember_201922 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,776 - root - INFO - 寫入 F:/temp\\3\\100_OrdMember_2019\\100_OrdMember_2019\\100_OrdMember_2019222 到壓縮包 F:/result\\rrrr.zip 成功 

2020-01-19 13:35:34,790 - root - INFO - 寫入 F:/temp\\3\\100_OrdMember_2019\\100_OrdMember_201911 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,792 - root - INFO - 寫入 F:/temp\\3\\100_OrdMember_2019\\100_OrdMember_2019111 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,794 - root - INFO - 寫入 F:/temp\\3\\100_OrdMember_2019\\tmp001.xls 到壓縮包 F:/result\\rrrr.zip 成功
2020-01-19 13:35:34,794 - root - INFO - ==========================壓縮完成==========================/<code>

使用pyinstaller打包

<code>pyinstaller -F -i .\\downRecordFile.ico recoderFileDownload.py/<code>

使用Python32位pyinstaller出來的是運行於32位Windows操作系統的程序，Python64位則適用於64位Windows操作系統。

閱讀更多 evilRat 的文章

關鍵字: JSON Guido 需求分析