我所經歷的大文件資料匯出（後台執行,自動產生）-php教程-PHP中文網

一、前言

記錄以前做的後台excel格式導出統計資訊的功能，也是最近同事問到了相關東西，一時之間竟忘了具體的細節，因此記錄一下；

知道，excel匯出資料的功能，後台幾乎是必須功能，一般都是點擊後，產生檔案然後自動下載，

如果是資料量小的話，一下子便可請求完成，從而下載到本地；

但是，如果資料量特別大的時候，頁面就必須一直在等待，直到寫入excel成功，

這樣就影響了後台使用者無法操作其他頁面，為此，對excel做了以下功能最佳化：

excel匯出分成兩部分內容：產生excel檔案和下載excel檔案

excel的檔案產生在程式後台執行，前端不必等待，可進行其他後台作業
增加下載檔案頁面，顯示excel檔案產生的進度，完成後，方可下載產生的excel檔案
檔案產生後，點選下載者可下載對應的檔案

二、產生檔案

產生excel檔案的方法有很多，暫不一一記錄，只是記錄這次的方法；

這裡用到了table的html格式，以及對應的excel的方法用office07打開的時候好像是亂碼，後面嘗試用csv格式文件，可還是亂碼，所以用了table的形式）

文件的開頭：

文件的結尾：

<spanmicrosoft yahei><span> 1</span><span>$struserdata</span> = <<<<span>Eof
</span><span> 2</span>         <html xmlns:o="urn:schemas-microsoft-com:office:office"
<span> 3</span>         xmlns:x="urn:schemas-microsoft-com:office:excel"
<span> 4</span>         xmlns="http://www.w3.org/TR/REC-html40">
<span> 5</span><span> 6</span>         <!DOCTYPE html <span>PUBLIC</span> "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<span> 7</span>         <html>
<span> 8</span>         <head>
<span> 9</span>             <meta http-equiv="Content-type" c />
<span>10</span>         <style>
<span>11</span>         </style>
<span>12</span>         </head>
<span>13</span>         <body>
<span>14</span>         <div align=center x:publishsource="Excel">
<span>15</span><span>16</span>         <table x:str border=1 cellpadding=0 cellspacing=0 width=100%>
<span>17</span> Eof;</spanmicrosoft>

登入後複製

View Code

三、讓程式在後台執行

場景：

excel ;

下載頁面可看到檔案產生的進度以及是否可下載狀態

load🠎 產生excel -- -> create_excel 方法

show_download方法中呼叫create_excel方法，而show_download 方法中，自己用了一下指令列執行程式的方式,
_ f
<spanmicrosoft yahei>1$struserdata = <<<Eof
2 </table>
3 </div>
4 </body>
5 </html>
6 Eof;</spanmicrosoft>
登入後複製
而在create_excel方法中：
需要填入以下程式碼：

rrrerreref

四、顯示檔案產生進度

但是怎麼顯示對應的文件產生進度呢，怎麼知道文件到底生成好了沒有呢？

這裡，我用到的方法是，在寫入資料檔案的時候data.xsl，每個資料檔案都會產生一個對應的檔案進度文件，暫且稱為flag_data.xsl;

第一次请求api的时候，根据返回的total总数，以及pagesize，确定要请求的次数count;

这样便可知道要请求api的次数（分页请求api），在写入数据文件的同时，同时写入进度文件flag_data.xsl;
数据格式大约是(以逗号分割)
1,5
2,5
...

然后显示文件进度的时候，读取进度文件，这样变可知道数据文件大体的进度

前端js处理时，几秒读取一次相应的方法（如果都100%进度，可停止请求方法），从而实现动态查看文件的生成进度

查看文件的进度方法：

<spanmicrosoft yahei><span> 1</span><span>public</span><span>function</span><span> execscript_process(){
</span><span> 2</span><span>$this</span>->load->library('smarty'<span>);
</span><span> 3</span><span>$file_arr_str</span> = <span>array</span><span>();
</span><span> 4</span><span>$file_arr_process</span> = <span>array</span><span>();
</span><span> 5</span><span>$file_arr_name</span> = <span>array</span><span>();
</span><span> 6</span><span>$file_arr</span> = <span>array</span><span>();
</span><span> 7</span><span>$refresh_flag</span> = 'yes'<span>;
</span><span> 8</span><span>$uid</span> = <span>$_REQUEST</span>['uid'<span>];
</span><span> 9</span><span>$url_dir</span> = STATISTIC_EXPORT_FILE_DIR.<span>$uid</span> .'/';<span>//</span><span>@todo</span><span>10</span><span>if</span>(!<span>is_dir</span>(<span>$url_dir</span><span>)){
</span><span>11</span>             @<span>mkdir</span>(<span>$url_dir</span>,0777<span>);
</span><span>12</span><span>        }
</span><span>13</span><span>$files</span> = <span>scandir</span>(<span>$url_dir</span><span>);
</span><span>14</span><span>15</span><span>if</span>(!<span>empty</span>(<span>$files</span><span>)){
</span><span>16</span><span>foreach</span> (<span>$files</span><span>as</span><span>$key</span> => <span>$value</span><span>) {
</span><span>17</span><span>if</span>(<span>$value</span>!='.' && <span>$value</span>!='..'<span>){
</span><span>18</span><span>if</span>(<span>substr</span>(<span>$value</span>, 0 , 5)=="flag_"<span>){
</span><span>19</span><span>$file_size</span> = <span>filesize</span>(<span>$url_dir</span> . <span>$value</span><span>);
</span><span>20</span><span>if</span>(!<span>empty</span>(<span>$file_size</span><span>)){
</span><span>21</span><span>$fhandle</span> = <span>fopen</span>(<span>$url_dir</span> . <span>$value</span>, 'rb+'<span>);
</span><span>22</span><span>fseek</span>(<span>$fhandle</span>, -1,<span> SEEK_END);
</span><span>23</span><span>$fstr</span> = ''<span>;
</span><span>24</span><span>while</span>((<span>$c</span> = <span>fgetc</span>(<span>$fhandle</span>)) !== <span>false</span><span>) {
</span><span>25</span><span>if</span>(<span>$c</span> == "\n" && <span>$fstr</span>) <span>break</span><span>;
</span><span>26</span><span>$fstr</span> = <span>$c</span> . <span>$fstr</span><span>;
</span><span>27</span><span>fseek</span>(<span>$fhandle</span>, -2,<span> SEEK_CUR);
</span><span>28</span><span>                            }
</span><span>29</span><span>fclose</span>(<span>$fhandle</span><span>);
</span><span>30</span><span>$fstr</span> = <span>trim</span>(<span>$fstr</span><span>);
</span><span>31</span><span>$fstr_arr_str</span> = <span>explode</span>(',', <span>$fstr</span><span>);
</span><span>32</span><span>$file_arr_process</span>[] = 100 * <span>number_format</span>(<span>$fstr_arr_str</span>[0]/<span>$fstr_arr_str</span>[1],2).'%'<span>;
</span><span>33</span><span>$file_arr_name</span>[] = <span>substr</span>(<span>$value</span>,5<span>);
</span><span>34</span><span>                         }
</span><span>35</span><span>                    }
</span><span>36</span><span>                }
</span><span>37</span><span>            }
</span><span>38</span><span>39</span><span>foreach</span> (<span>$file_arr_process</span><span>as</span><span>$key</span> => <span>$value</span><span>) {
</span><span>40</span><span>if</span>(<span>$value</span> != '100%'<span>){
</span><span>41</span><span>$refresh_flag</span> = 'no'<span>;
</span><span>42</span><span>break</span><span>;
</span><span>43</span><span>                }
</span><span>44</span><span>            }
</span><span>45</span><span>        }
</span><span>46</span><span>47</span><span>$file_arr</span> = <span>array</span><span>(
</span><span>48</span>             'process' => <span>$file_arr_process</span>,
<span>49</span>             'name' => <span>$file_arr_name</span>,
<span>50</span>             'refresh_flag' => <span>$refresh_flag</span><span>51</span><span>            );
</span><span>52</span><span>$file_arr_json</span> = json_encode(<span>$file_arr</span><span>);
</span><span>53</span><span>echo</span><span>$file_arr_json</span><span>;
</span><span>54</span>     }</spanmicrosoft>

登入後複製

View Code

五、下载文件

文件的下载就好说了，既然已经都生成成功，下载的方法如下：

<spanmicrosoft yahei><span> 1</span><span>public</span><span>function</span><span> execscript_download(){
</span><span> 2</span><span>$filename</span> = <span>$_REQUEST</span>['filename'<span>];
</span><span> 3</span><span>$uid</span> = <span>$_REQUEST</span>['uid'<span>];
</span><span> 4</span><span>$file_dir</span> = STATISTIC_EXPORT_FILE_DIR.<span>$uid</span>.'/'.<span>$filename</span><span>;
</span><span> 5</span><span>if</span> (!<span>file_exists</span>(<span>$file_dir</span><span>)){
</span><span> 6</span><span>header</span>("Content-type: text/html; charset=utf-8"<span>);
</span><span> 7</span><span>echo</span> "File not found!"<span>;
</span><span> 8</span><span>exit</span><span>; 
</span><span> 9</span>         } <span>else</span><span> {
</span><span>10</span><span>ini_set</span>("memory_limit","500M"<span>); 
</span><span>11</span><span>header</span>('Content-Description: File Transfer'<span>);
</span><span>12</span><span>header</span>('Content-Type: application/octet-stream'<span>);
</span><span>13</span><span>header</span>('Content-Disposition: attachment; filename='.<span>basename</span>(<span>$file_dir</span><span>));
</span><span>14</span><span>header</span>('Content-Transfer-Encoding: binary'<span>);
</span><span>15</span><span>header</span>('Expires: ' . <span>gmdate</span>('D, d M Y H:i:s') . ' GMT'<span>);
</span><span>16</span><span>header</span>('Cache-Control: must-revalidate,post-check=0, pre-check=0'<span>);
</span><span>17</span><span>header</span>('Pragma: public'<span>);
</span><span>18</span><span>header</span>('Content-Length: ' . <span>filesize</span>(<span>$file_dir</span><span>));
</span><span>19</span><span>readfile</span>(<span>$file_dir</span><span>);
</span><span>20</span><span>        }
</span><span>21</span><span>22</span>     }</spanmicrosoft>

登入後複製

六、上线后出现的问题

本地本来已经测试完毕，可上线后，却出现了奇怪的问题；

现象描述：

当在后台点击生成文件，跳转到下载页的时候，因为下载页是显示文件进度的页面，
竟然出现有时候有刚刚点击的文件进度，有时候没有，就感觉没有生成相应的文件一样；

解决方法：

因为数据文件和进度文件都是生成在程序的某个文件夹file中，所以读取的时候都是读取的文件夹下的文件，从而判断显示进度；

后面才知道，由于后台程序有两台服务器，导致读取以及下载的时候找不到相应的文件夹，两个服务器相应的文件夹弄个共享目录就可以了

七、相应的后续优化

由于下载的文件多了，导致文件夹下的文件越来越多，而原来生成的文件是没有价值的，所以加了个定期删除文件的功能，只保留近七天的文件

当然可以用crontab，只不过我比较懒，是在点击生成文件的时候，判断了一下文件夹中的过期文件，从而删除

<spanmicrosoft yahei><span> 1</span><span>public</span><span>function</span><span> execscript_process_show(){
</span><span> 2</span><span>$this</span>->load->library('smarty'<span>);
</span><span> 3</span><span>$uid</span> = <span>$_REQUEST</span>['uid'<span>];
</span><span> 4</span><span>$url_dir</span> = STATISTIC_EXPORT_FILE_DIR.<span>$uid</span> .'/';<span>//</span><span>@todo</span><span> 5</span><span>if</span>(!<span>is_dir</span>(<span>$url_dir</span><span>)){
</span><span> 6</span>             @<span>mkdir</span>(<span>$url_dir</span>,0777<span>);
</span><span> 7</span><span>        }        
</span><span> 8</span><span>$files</span> = <span>scandir</span>(<span>$url_dir</span><span>);
</span><span> 9</span><span>if</span>(!<span>empty</span>(<span>$files</span><span>)){
</span><span>10</span><span>foreach</span> (<span>$files</span><span>as</span><span>$key</span> => <span>$value</span><span>) {
</span><span>11</span><span>if</span>(<span>$value</span>!='.' && <span>$value</span>!='..'<span>){
</span><span>12</span><span>foreach</span> (<span>$files</span><span>as</span><span>$key</span> => <span>$value</span><span>) {
</span><span>13</span><span>if</span>(<span>$value</span>!='.' && <span>$value</span>!='..'<span>){
</span><span>14</span><span>if</span>(<span>substr</span>(<span>$value</span>, 0 , 5)!="flag_"<span>){
</span><span>15</span><span>$filenamedate</span> = <span>substr</span>(<span>$value</span>, 0,10<span>);
</span><span>16</span><span>$today</span> = <span>date</span>('Y-m-d',<span>time</span><span>());
</span><span>17</span><span>$filenamedate</span> = <span>date</span>('Y-m-d',<span>strtotime</span>(<span>$filenamedate</span>)+(STATISTIC_FILE_EXPIRE_DAY-1)*24*3600<span>);
</span><span>18</span><span>if</span>(<span>$today</span>><span>$filenamedate</span>){<span>//</span><span>文件过期</span><span>19</span>                                     @<span>unlink</span>(<span>$url_dir</span> . <span>$value</span><span>);
</span><span>20</span>                                     @<span>unlink</span>(<span>$url_dir</span> . 'flag_' . <span>$value</span><span>);
</span><span>21</span><span>                                }
</span><span>22</span><span>                            }
</span><span>23</span><span>                        }
</span><span>24</span><span>                    }                    
</span><span>25</span><span>                }
</span><span>26</span><span>            }
</span><span>27</span><span>        }
</span><span>28</span><span>29</span><span>$this</span>->smarty->assign('uid',<span>$uid</span><span>);
</span><span>30</span><span>$this</span>->smarty->display('interact/statistic/execscript.tpl'<span>);
</span><span>31</span>     }</spanmicrosoft>

登入後複製

八、后记

大文件的导出大体就是这个样子，欢迎大家吐槽，共同交流；

当时在用命令行执行方法的时候，也参考了一下相应的资料，记录一下；

<spanmicrosoft yahei>http://blog.csdn.net/yysdsyl/article/details/4636457

http://www.codesky.net/article/201202/163385.html

http://www.cnblogs.com/zdz8207/p/3765567.html

http://blog.163.com/mojian20040228@126/blog/static/4112219320097300922992/

http://php.net/manual/en/features.commandline.php

http://blog.csdn.net/yangjun07167/article/details/5603425

http://blog.csdn.net/yunsongice/article/details/5445448

http://www.cppblog.com/amazon/archive/2011/12/01/161281.aspx

http://blog.51yip.com/tag/proc_open

http://www.justwinit.cn/post/1418/

http://limboy.me/tech/2010/12/05/php-async.html</spanmicrosoft>

登入後複製

以上就介绍了我所经历的大文件数据导出（后台执行,自动生成），包括了方面的内容，希望对PHP教程有兴趣的朋友有所帮助。