Ubuntu 16.04 上使用Nginx、 Gunicorn、

本文记录Ubuntu 16.04 上使用Nginx、 Gunicorn、supervisor 部署django项目

准备工作:

• 创建一个django项目
• 创建一个python虚拟环境
• 激活虚拟环境
  这里跳过准备工作，假设我已经有了一个项目DjangoBlog和一个虚拟环境alpfaceblog

安装配置gunicorn

• 首先切换到项目的虚拟环境

workon alpfaceblog

• 在虚拟环境中安装gunicorn

pip install django gunicorn

• 把 gunicorn 加入项目settings.py中的 INSTALLED_APPS

INSTALLED_APPS = (
    # ...
    'gunicorn',
)

• 测试gunicorn是否可以正常工作

cd /var/www/Blog
sudo gunicorn -b 0.0.0.0:8888 --worker-class=gevent DjangoBlog.wsgi:application

• 配置生产环境下的gunicorn
  创建一个bash脚本用于快速启动gunicorn的
  在项目目录下创建一个文件夹bin，然后创建gunicorn_start文件

sudo vim  gunicorn_start

添加以下内容,注意路径需要根据实际情况替换

#!/bin/bash

NAME="DjangoBlog" # 项目的名称
DJANGODIR=/var/www/Blog # 项目所在的目录
SOCKFILE=/var/www/Blog/run/gunicorn.sock # 启动gunicorn.sock使用UNIX套接字
USER=root # the user to run as
GROUP=root # the group to run as
NUM_WORKERS=3 # how many worker processes should Gunicorn spawn
DJANGO_SETTINGS_MODULE=DjangoBlog.settings # which settings file should Django use
DJANGO_WSGI_MODULE=DjangoBlog.wsgi # WSGI module name

echo "Starting $NAME as `whoami`"

# 启动虚拟环境的路径
cd $DJANGODIR
source /home/parallels/.virtualenvs/alpfaceblog/bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH

# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR

# Start your Django Unicorn
# gunicorn 安装在虚拟环境下的完整路径
exec /home/parallels/.virtualenvs/alpfaceblog/bin/gunicorn ${DJANGO_WSGI_MODULE}:application \
--name $NAME \
--workers $NUM_WORKERS \
--user=$USER --group=$GROUP \
--bind=unix:$SOCKFILE \
--log-level=debug \
--log-file=-

添加可执行权限： sudo chmod u+x bin/gunicorn_start
启动gunicorn

 ./bin/gunicorn_start

安装并配置Supervisor

Superviosr是一个进程监管的工具。简而言之，Superviosr可以保证你的程序在服务器开机时自动启动以及程序意外终止时重新启动。

• 安装

sudo apt-get install supervisor

• 配置

supervisor的配置文件默认从/etc/supervisor/conf.d中读取

cd /etc/supervisor/conf.d

创建并编辑一个supervisor的配置文件

sudo vim alpfaceblog.conf 

添加下面内容，注意路径需要根据实际情况替换

[program:DjangoBlog]
command = /var/www/Blog/bin/django_start
user = root
autostart=true
autorestart=true

redirect_stderr = true
stdout_logfile = /root/logs/blog/robot.log
stderr_logfile=/root/logs/blog/err.log  

手动创建日志目录:

mkdir -p /root/logs/blog

上面一步我们使用了django_start，这里我们在项目的bin目录下创建下它, 路径需要根据实际情况替换：

sudo vim django_start

#!/bin/bash

NAME="DjangoBlog" # Name of the application
DJANGODIR=/var/www/Blog # Django project directory
SOCKFILE=/var/www/Blog/run/gunicorn.sock # we will communicte using this unix socket
USER=root # the user to run as
GROUP=root # the group to run as
NUM_WORKERS=3 # how many worker processes should Gunicorn spawn
DJANGO_SETTINGS_MODULE=DjangoBlog.settings # which settings file should Django use
DJANGO_WSGI_MODULE=DjangoBlog.wsgi # WSGI module name

echo "Starting $NAME as `whoami`"

# Activate the virtual environment
cd $DJANGODIR
source /home/parallels/.virtualenvs/alpfaceblog/bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH

# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR

# Start your Django Unicorn
# Programs meant to be run under supervisor should not daemonize themselves (do not use --daemon)
exec /home/parallels/.virtualenvs/alpfaceblog/bin/gunicorn ${DJANGO_WSGI_MODULE}:application \
--name $NAME \
--workers $NUM_WORKERS \
--user=$USER --group=$GROUP \
--bind=unix:$SOCKFILE \
--log-level=debug \
--log-file=-

通过supervisorctl工具来启用这些设置:

sudo supervisorctl update
sudo supervisorctl reload  

这里我遇到了启动错误信息:

pkg_resources.DistributionNotFound: The 'supervisor==3.2.0' distribution was

解决方法:
这是由于我把ubuntu的python默认环境改成3.5的了，其实再改回2.7就好了

切换Python版本
sudo update-alternatives --install /usr/bin/python python /usr/bin/python2 200
sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 100

安装并配置nginx

• 安装nginx

sudo apt-get install nginx

ubantu安装完Nginx后，文件结构大致为：
　　所有的配置文件都在 /etc/nginx下；
　　启动程序文件在 /usr/sbin/nginx下；
　　日志文件在 /var/log/nginx/下，分别是access.log和error.log；
　　并且在 /etc/init.d下创建了启动脚本nginx。

sudo /etc/init.d/nginx start # 启动
sudo /etc/init.d/nginx stop # 停止
sudo /etc/init.d/nginx restart # 重启

• 配置nginx

配置Nginx 为我们的Django应用创建一个配置文件/etc/nginx/sites-available/alpfaceblog.conf

sudo vim alpfaceblog.conf

本文记录Ubuntu 16.04 上使用Nginx、 Gunicorn、supervisor 部署django项目

准备工作:

• 创建一个django项目
• 创建一个python虚拟环境
• 激活虚拟环境
  这里跳过准备工作，假设我已经有了一个项目DjangoBlog和一个虚拟环境alpfaceblog

创建数据库

登录mysql创建一个数据库,这里创建一个名称为blog数据库:

mysql -uroot -p

create database blog charset=utf8;

配置DjangoBlog/setting.py 中数据库 相关配置，如下所示：

     DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.mysql',
            'NAME': 'blog',
            'USER': 'root',
            'PASSWORD': 'root',
            'HOST': 'host',
            'PORT': 3306,
        }
    }

安装配置gunicorn

• 首先切换到项目的虚拟环境

workon alpfaceblog

• 创建数据库

python manage.py makemigrations
python manage.py migrate

• 创建超级用户

python manage.py createsuperuser

• 收集静态文件

python manage.py collectstatic --noinput
python manage.py compress --force

• 在虚拟环境中安装gunicorn

pip install django gunicorn

• 把 gunicorn 加入项目settings.py中的 INSTALLED_APPS

INSTALLED_APPS = (
    # ...
    'gunicorn',
)

• 测试gunicorn是否可以正常工作

cd /var/www/Blog
sudo gunicorn -b 0.0.0.0:8888 --worker-class=gevent DjangoBlog.wsgi:application

• 配置生产环境下的gunicorn
  创建一个bash脚本用于快速启动gunicorn的
  在项目目录下创建一个文件夹bin，然后创建gunicorn_start文件

sudo vim  gunicorn_start

添加以下内容,注意路径需要根据实际情况替换

#!/bin/bash

NAME="DjangoBlog" # 项目的名称
DJANGODIR=/var/www/Blog # 项目所在的目录
SOCKFILE=/var/www/Blog/run/gunicorn.sock # 启动gunicorn.sock使用UNIX套接字
USER=root # the user to run as
GROUP=root # the group to run as
NUM_WORKERS=3 # how many worker processes should Gunicorn spawn
DJANGO_SETTINGS_MODULE=DjangoBlog.settings # which settings file should Django use
DJANGO_WSGI_MODULE=DjangoBlog.wsgi # WSGI module name

echo "Starting $NAME as `whoami`"

# 启动虚拟环境的路径
cd $DJANGODIR
source /home/parallels/.virtualenvs/alpfaceblog/bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH

# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR

# Start your Django Unicorn
# gunicorn 安装在虚拟环境下的完整路径
exec /home/parallels/.virtualenvs/alpfaceblog/bin/gunicorn ${DJANGO_WSGI_MODULE}:application \
--name $NAME \
--workers $NUM_WORKERS \
--user=$USER --group=$GROUP \
--bind=unix:$SOCKFILE \
--log-level=debug \
--log-file=-

添加可执行权限： sudo chmod u+x bin/gunicorn_start
启动gunicorn

 ./bin/gunicorn_start

安装并配置Supervisor

Superviosr是一个进程监管的工具。简而言之，Superviosr可以保证你的程序在服务器开机时自动启动以及程序意外终止时重新启动。

• 安装

sudo apt-get install supervisor

• 配置

supervisor的配置文件默认从/etc/supervisor/conf.d中读取

cd /etc/supervisor/conf.d

创建并编辑一个supervisor的配置文件

sudo vim alpfaceblog.conf 

添加下面内容，注意路径需要根据实际情况替换

[program:DjangoBlog]
command = /var/www/Blog/bin/django_start
user = root
autostart=true
autorestart=true

redirect_stderr = true
stdout_logfile = /root/logs/blog/robot.log
stderr_logfile=/root/logs/blog/err.log  

手动创建日志目录:

mkdir -p /root/logs/blog

上面一步我们使用了django_start，这里我们在项目的bin目录下创建下它, 路径需要根据实际情况替换：

sudo vim django_start

#!/bin/bash

NAME="DjangoBlog" # Name of the application
DJANGODIR=/var/www/Blog # Django project directory
SOCKFILE=/var/www/Blog/run/gunicorn.sock # we will communicte using this unix socket
USER=root # the user to run as
GROUP=root # the group to run as
NUM_WORKERS=3 # how many worker processes should Gunicorn spawn
DJANGO_SETTINGS_MODULE=DjangoBlog.settings # which settings file should Django use
DJANGO_WSGI_MODULE=DjangoBlog.wsgi # WSGI module name

echo "Starting $NAME as `whoami`"

# Activate the virtual environment
cd $DJANGODIR
source /home/parallels/.virtualenvs/alpfaceblog/bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH

# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR

# Start your Django Unicorn
# Programs meant to be run under supervisor should not daemonize themselves (do not use --daemon)
exec /home/parallels/.virtualenvs/alpfaceblog/bin/gunicorn ${DJANGO_WSGI_MODULE}:application \
--name $NAME \
--workers $NUM_WORKERS \
--user=$USER --group=$GROUP \
--bind=unix:$SOCKFILE \
--log-level=debug \
--log-file=-

通过supervisorctl工具来启用这些设置:

sudo supervisorctl update
sudo supervisorctl reload  

这里我遇到了启动错误信息:

pkg_resources.DistributionNotFound: The 'supervisor==3.2.0' distribution was

解决方法:
这是由于我把ubuntu的python默认环境改成3.5的了，其实再改回2.7就好了

切换Python版本
sudo update-alternatives --install /usr/bin/python python /usr/bin/python2 200
sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 100

安装并配置nginx

• 安装nginx

sudo apt-get install nginx

ubantu安装完Nginx后，文件结构大致为：
　　所有的配置文件都在 /etc/nginx下；
　　启动程序文件在 /usr/sbin/nginx下；
　　日志文件在 /var/log/nginx/下，分别是access.log和error.log；
　　并且在 /etc/init.d下创建了启动脚本nginx。

sudo /etc/init.d/nginx start # 启动
sudo /etc/init.d/nginx stop # 停止
sudo /etc/init.d/nginx restart # 重启

• 配置nginx

配置Nginx 为我们的Django应用创建一个配置文件/etc/nginx/sites-available/alpfaceblog.conf

sudo vim alpfaceblog.conf

添加以下内容

upstream my_server {
    server unix:/var/www/Blog/run/gunicorn.sock fail_timeout=0;
}

server {

    server_name localhost;
    
    upstream my_server {
    server unix:/var/www/Blog/run/gunicorn.sock fail_timeout=0;
}

server {
    listen 80;
    server_name localhost;

    large_client_header_buffers 4 16k;
    client_max_body_size 300m;
    client_body_buffer_size 128k;
    proxy_connect_timeout 600;
    proxy_read_timeout 600;
    proxy_send_timeout 600;
    proxy_buffer_size 64k;
    proxy_buffers   4 32k;
    proxy_busy_buffers_size 64k;
    proxy_temp_file_write_size 64k;

    root /var/www/Blog;

    keepalive_timeout 70;
    access_log /var/log/nginx/django_access.log;
    error_log /var/log/nginx/django_error.log;

    location /static {
          expires max;
          alias /var/www/Blog/collectedstatic/;
    }

    location /media  {
        alias /var/www/Blog/media/;
    }
    location / {
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        if (!-f $request_filename) {
            proxy_pass http://my_server;
            break;
        }
    }

}
    root /var/www/Blog;

    listen 80;

    keepalive_timeout 70;
    access_log /var/log/nginx/django_access.log;
    error_log /var/log/nginx/django_error.log;

    location /static/ {
          expires max;
          alias /var/www/Blog/collectedstatic/;
    }

    location / {
         proxy_set_header Host $host;
         proxy_set_header X-Real-IP $remote_addr;

         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;


        if (!-f $request_filename) {
            proxy_pass http://my_server;
            break;
        }
    }

}

• 让nginx的配置文件生效

 sudo ln -s  /etc/nginx/sites-available/alpfaceblog.conf   /etc/nginx/sites-enabled/alpfaceblog.conf

• 重启nginx服务器

sudo service nginx restart

問題:

nginx遇见的问题

1.最近在访问部分页面时，速度经常非常慢，导致报502 Bad Gateway错误
解决方法:
查看nginx的错误日志，我是配置在error_log /var/log/nginx/django_error.log;下

日志中我发现了两个问题，第一个问题:

ubuntu@ip-172-31-17-237:/var/log/nginx$ cat django_error.log
2018/02/24 09:10:43 [error] 1132#1132: *72 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 66.249.64.22, server: www.alpface.com, request: "GET /article/2018/1/20/1.html HTTP/1.1", upstream: "http://unix:/var/www/Blog/run/gunicorn.sock/article/2018/1/20/1.html", host: "www.alpface.net"
2018/02/24 09:22:42 [error] 1132#1132: *87 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 222.249.170.75, server: www.alpface.com, request: "POST /admin/blog/article/add/ HTTP/1.1", upstream: "http://unix:/var/www/Blog/run/gunicorn.sock/admin/blog/article/add/", host: "alpface.com", referrer: "http://alpface.com/admin/blog/article/add/"

通过网友的各种尝试，我决定试一试，最终解决了，解决方法:
这是由于，NGINX反向代理的超时报错:

ubuntu@ip-172-31-17-237:/var/log/nginx$ cd /etc/nginx/sites-available/
ubuntu@ip-172-31-17-237:/etc/nginx/sites-available$ sudo vi alpfaceblog.conf 

在网站的配置文件中添加以下，并重启nginx

server {
listen 80;
server_name localhost;

    large_client_header_buffers 4 16k;  
    client_max_body_size 300m;  
    client_body_buffer_size 128k;  
    proxy_connect_timeout 600;  
    proxy_read_timeout 600;  
    proxy_send_timeout 600;  
    proxy_buffer_size 64k;  
    proxy_buffers   4 32k;  
    proxy_busy_buffers_size 64k;  
    proxy_temp_file_write_size 64k;  

    #.............................  
}  

重启：`sudo /etc/init.d/nginx restart`

第二个问题：
查看错误日志可以看到：`/var/www/Blog/media//`, media后面多了一个/, 这是由于nginx配置文件中的资源文件路径写错了导致的，修改下我的配置文件，重启nginx解决；

ubuntu@ip-172-31-17-237:/var/log/nginx$ cat django_error.log
2018/03/05 06:44:04 [error] 3493#3493: *1 upstream prematurely closed connection while reading response header from upstream, client: 222.249.170.75, server: localhost, request: "GET /article/2018/2/28/38.html HTTP/1.1", upstream: "http://unix:/var/www/Blog/run/gunicorn.sock:/article/2018/2/28/38.html", host: "www.alpface.com", referrer: "http://www.alpface.com/login/?next=/article/2018/2/28/38.html"
2018/03/05 06:51:16 [error] 3493#3493: *33 directory index of "/var/www/Blog/media//" is forbidden, client: 222.249.170.75, server: localhost, request: "GET /media/ HTTP/1.1", host: "www.alpface.com", referrer: "http://www.alpface.com/admin/article/edit/?aid=32"
2018/03/05 06:51:26 [error] 3493#3493: *32 open() "/var/www/Blog/collectedstatic//blog/img/favicon.ico" failed (2: No such file or directory), client: 222.249.170.75, server: localhost, request: "GET /static/blog/img/favicon.ico HTTP/1.1", host: "www.alpface.com", referrer: "http://www.alpface.com/admin/article/edit/?aid=32"
2018/03/05 06:56:05 [error] 3493#3493: *39 directory index of "/var/www/Blog/media//" is forbidden, client: 222.249.170.75, server: localhost, request: "GET /media/ HTTP/1.1", host: "www.alpface.com", referrer: "http://www.alpface.com/admin/article/list/"
2018/03/05 06:56:09 [error] 3493#3493: *41 directory index of "/var/www/Blog/media//" is forbidden, client: 222.249.170.75, server: localhost, request: "GET /media/ HTTP/1.1", host: "www.alpface.com", referrer: "http://www.alpface.com/admin/message/comment/"
2018/03/05 06:56:11 [error] 3493#3493: *41 directory index of "/var/www/Blog/media//" is forbidden, client: 222.249.170.75, server: localhost, request: "GET /media/ HTTP/1.1", host: "www.alpface.com", referrer: "http://www.alpface.com/admin/message/os/"
2018/03/05 06:56:12 [error] 3493#3493: *41 directory index of "/var/www/Blog/media//" is forbidden, client: 222.249.170.75, server: localhost, request: "GET /media/ HTTP/1.1", host: "www.alpface.com", referrer: "http://www.alpface.com/admin/message/comment/"
2018/03/05 06:56:15 [error] 3493#3493: *39 directory index of "/var/www/Blog/media//" is forbidden, client: 222.249.170.75, server: localhost, request: "GET /media/ HTTP/1.1", host: "www.alpface.com", referrer: "http://www.alpface.com/admin/visitor/"
2018/03/05 06:56:18 [error] 3493#3493: *41 directory index of "/var/www/Blog/media//" is forbidden, client: 222.249.170.75, server: localhost, request: "GET /media/ HTTP/1.1", host: "www.alpface.com", referrer: "http://www.alpface.com/admin/link/"
2018/03/05 06:56:23 [error] 3493#3493: *41 directory index of "/var/www/Blog/media//" is forbidden, client: 222.249.170.75, server: localhost, request: "GET /media/ HTTP/1.1", host: "www.alpface.com", referrer: "http://www.alpface.com/admin/profile/"
2018/03/05 06:56:37 [error] 3493#3493: *41 directory index of "/var/www/Blog/media//" is forbidden, client: 222.249.170.75, server: localhost, request: "GET /media/ HTTP/1.1", host: "www.alpface.com", referrer: "http://www.alpface.com/admin/profile/"
2018/03/05 06:56:40 [error] 3493#3493: *41 directory index of "/var/www/Blog/media//" is forbidden, client: 222.249.170.75, server: localhost, request: "GET /media/ HTTP/1.1", host: "www.alpface.com", referrer: "http://www.alpface.com/admin/profile/"
2018/03/05 06:56:41 [error] 3493#3493: *39 directory index of "/var/www/Blog/media//" is forbidden, client: 222.249.170.75, server: localhost, request: "GET /media/ HTTP/1.1", host: "www.alpface.com", referrer: "http://www.alpface.com/admin/link/"
2018/03/05 06:56:43 [error] 3493#3493: *39 directory index of "/var/www/Blog/media//" is forbidden, client: 222.249.170.75, server: localhost, request: "GET /media/ HTTP/1.1", host: "www.alpface.com", referrer: "http://www.alpface.com/admin/profile/"
2018/03/05 06:56:45 [error] 3493#3493: *39 directory index of "/var/www/Blog/media//" is forbidden, client: 222.249.170.75, server: localhost, request: "GET /media/ HTTP/1.1", host: "www.alpface.com", referrer: "http://www.alpface.com/admin/article/add/"
2018/03/05 06:56:52 [error] 3493#3493: *39 directory index of "/var/www/Blog/media//" is forbidden, client: 222.249.170.75, server: localhost, request: "GET /media/ HTTP/1.1", host: "www.alpface.com", referrer: "http://www.alpface.com/admin/"

##### 执行数据库迁移时遇见的问题
1. 當修改或新增model後, 執行python manage.py makemigrations,未生成遷移:
解決方法:
刪除每個app文件下migrations文件中除了__init__.py的所有文件,然後再執行`python manage.py makemigrations`重新生成遷移

2. 數據庫遷移時遇見的問題:
錯誤信息:

django.db.utils.InternalError: (1054, "Unknown column 'name' in 'django_content_type'")

解決方法:
在数据库中手动添加没有创建的字段

alter table django_content_type add column name varchar(10)

3. 遷移數據庫python manage.py migrate時出錯:

django.db.utils.InternalError: (1050, "Table 'auth_permission' already exists")

解決方法:

python manage.py migrate --fake

但是此種方法會忽略一些錯誤,導致最終還是不能解決我們的遷移的問題


總結:
我的blog項目是在我的本地電腦ubuntu系統上開發並測試的,然後提交到github上,最後部署到亞馬遜服務器時,通過git命令clone到服務器上;
期間我在本地修改或創建了一些model,這時我在本地生成遷移和遷移時都沒有,但是在服務端遷移總是報錯,導致遷移失敗;
原因:在網上查找各種資料最終都沒有解決,最後我發覺項目的.gitignore中忽略了migrations文件,而每次改變model或新增model時,都會生成一個000n_.py文件,每次改變n都會加1,
此時我的數據庫也會在遷移時發生改變,而當我在服務器執行遷移時,服務器端沒有migrations文件,每次遷移都會重新生成,那麼和本地執行時機不同,所以會需要每次執行遷移時,會根據migrations執行的結果也不同;
解決方法:當然我們可以在gitignore中取消這些忽略文件, 另外當我們在服務器段執行遷移遇到文件時,我們可以把本地的0001_initial.py中的復制到服務端對應的這個文件再執行`python manage.py makemigrations`和`python manage.py migrate`