My notes on NGINX administration basics, tips & tricks, caveats, and gotchas.
Hi-diddle-diddle, he played on his
fiddle and danced with lady pigs.
Number three said, "Nicks on tricks!
I'll build my house with EN-jin-EKS!".
The Three Little Pigs: Who's Afraid of the Big Bad Wolf?
Before you start playing with NGINX please read an official Beginner’s Guide. It's a great introduction for everyone.
Nginx (/ˌɛndʒɪnˈɛks/ EN-jin-EKS, stylized as NGINX or nginx) is an HTTP and reverse proxy server, a mail proxy server, and a generic TCP/UDP proxy server. It is originally written by Igor Sysoev. For a long time, it has been running on many heavily loaded Russian sites including Yandex, Mail.Ru, VK, and Rambler.
NGINX is a fast, light-weight and powerful web server that can also be used as a fast HTTP reverse proxy, reliable load balancer and high performance caching server. Generally it provides the core of complete web stacks.
These essential documents should be the main source of knowledge for you:
In addition, I would like to recommend two great articles focuses on the concept of the HTTP protocol:
This handbook is a collection of rules, notes and papers, best practices and recommendations collected and used by me (also in production environments but not only). Many of these refer to external resources.
I've never found one guide that covers the most important things about NGINX, and around NGINX. Of course, we have official documentation - it's probably the best place for us.
I think, however, there hasn't been a truly in-depth cheatsheet which describe a variety of configurations and important cross-cutting topics for HTTP servers. That's why I created this repository to help us to configure high performing NGINX web and proxy servers that are fast, secure and stable. I still have a lot to improve and to do.
Throughout this handbook you will explore the many features of NGINX and how to use them. This guide is fairly comprehensive, and touches a lot of the functions (e.g. security, performance) of NGINX.
If you do not have the time to read hundreds of articles this multipurpose handbook may be useful. I created it in the hope that it will be useful especially for System Administrators and WebOps. I hope you enjoy it.
Before you start remember about the two most important things:
Do not follow guides just to get 100% of something. Think about what you actually do at your server!
These guidelines provides recommendations for very restrictive setup.
A real community, however, exists only when its members interact in a meaningful way that deepens their understanding of each other and leads to learning.
If you find something which doesn't make sense, or something doesn't seem right, please make a pull request and please add valid and well-reasoned explanations about your changes or comments.
Before adding a pull request, please see the contributing guidelines.
If this project is useful and important for you, you can bring positive energy by giving some good words or supporting this project. Thank you!
New chapters:
Existing chapters:
Other stuff:
Many of these recipes have been applied to the configuration of my private website.
An example configuration is in configuration examples chapter. It's also based on this version of printable high-res hardening cheatsheet.
Read about SSL Labs grading here (SSL Labs Grading 2018).
Short SSL Labs grades explanation: A+ is clearly the desired grade, both A and B grades are acceptable and result in adequate commercial security. The B grade, in particular, may be applied to configurations designed to support very wide audiences (for old clients).
I finally got A+ grade and following scores:
Read about Mozilla Observatory here.
I also got the highest note from Mozilla:
I created printable posters with hardening cheatsheets (High-Res 5000x8200) based on these recipes:
For
*.xcf
and
A+ with all 100%’s on @ssllabs and 120/100 on @mozilla observatory:
It provides the highest scores of the SSL Labs test. Setup is very restrictive with 4096-bit private key, only TLS 1.2 and also modern strict TLS cipher suites (non 128-bits).
A+ on @ssllabs and 120/100 on @mozilla observatory with TLS 1.3 support:
It provides less restrictive setup with 2048-bit private key, TLS 1.2 and 1.3 and also modern strict TLS cipher suites (128/256-bits). The final grade is also in line with the industry standards. Recommend using this configuration.
This checklist contains 44 rules from this handbook.
Generally, I think that each of these principles is important and should be considered. I tried, however, to separate them into three levels of priority.
RULE | CHAPTER | SEVERITY |
---|---|---|
Define the listen directives explicitly with address:port pair Prevents soft mistakes which may be difficult to debug. |
Base Rules | |
Prevent processing requests with undefined server names It also protects against configuration errors and don't pass traffic to incorrect backends |
Base Rules | |
Force all connections over TLS Protect your website especially for handle sensitive communications. |
Base Rules | |
Keep NGINX up-to-date Use newest NGINX package to fix a vulnerabilities, bugs and use new features. |
Hardening | |
Run as an unprivileged user Use the principle of least privilege. This way only master process runs as root. |
Hardening | |
Protect sensitive resources Hidden directories and files should never be web accessible. |
Hardening | |
Use min. 2048-bit private keys 2048 bits private keys are sufficient for commercial use. |
Hardening | |
Keep only TLS 1.2 and TLS 1.3 Use TLS with modern cryptographic algorithms and without protocol weaknesses. |
Hardening | |
Use only strong ciphers Use only strong and not vulnerable cipher suites. |
Hardening | |
Use more secure ECDH Curve Use ECDH Curves with according to NIST recommendations. |
Hardening | |
Use strong Key Exchange | Hardening | |
Defend against the BEAST attack The server ciphers should be preferred over the client ciphers. |
Hardening | |
Mitigation of CRIME/BREACH attacks Disable HTTP compression or compress only zero sensitive content. |
Hardening | |
HTTP Strict Transport Security Tell browsers that it should only be accessed using HTTPS, instead of using HTTP. |
Hardening | |
Reduce XSS risks (Content-Security-Policy) CSP is best used as defense-in-depth. It reduces the harm that a malicious injection can cause. |
Hardening | |
Control the behavior of the Referer header (Referrer-Policy) The default behaviour of referrer leaking puts websites at risk of privacy and security breaches. |
Hardening | |
Provide clickjacking protection (X-Frame-Options) | Hardening | |
Prevent some categories of XSS attacks (X-XSS-Protection) Prevents to render pages if a potential XSS reflection attack is detected. |
Hardening | |
Prevent Sniff Mimetype middleware (X-Content-Type-Options) Tells browsers not to sniff MIME types. |
Hardening | |
Reject unsafe HTTP methods Only allow the HTTP methods for which you, in fact, provide services. |
Hardening | |
Organising Nginx configuration | Base Rules | |
Format, prettify and indent your Nginx code | Base Rules | |
Use HTTP/2 HTTP/2 will make our applications faster, simpler, and more robust. |
Performance | |
Maintaining SSL sessions | Performance | |
Use exact names in server_name directive where possible | Performance | |
Avoid checks server_name with if directive | Performance | |
Disable unnecessary modules Limits vulnerabilities, improve performance and memory efficiency. |
Hardening | |
Hide Nginx version number | Hardening | |
Hide Nginx server signature | Hardening | |
Hide upstream proxy headers | Hardening | |
Use only the latest supported OpenSSL version | Hardening | |
Deny the use of browser features (Feature-Policy) | Hardening | |
Control Buffer Overflow attacks | Hardening | |
Mitigating Slow HTTP DoS attack (Closing Slow Connections) | Hardening | |
Enable DNS CAA Policy | Others | |
Separate listen directives for 80 and 443 | Base Rules | |
Use only one SSL config for specific listen directive | Base Rules | |
Use geo/map modules instead allow/deny | Base Rules | |
Drop the same root inside location block | Base Rules | |
Adjust worker processes | Performance | |
Make an exact location match to speed up the selection process | Performance | |
Use limit_conn to improve limiting the download speed | Performance | |
Tweak passive health checks | Load Balancing | |
Don't disable backends by comments, use down parameter | Load Balancing | |
Define security policies with security.txt | Others |
These books are probably pay or free. They can be official and unofficial.
Authors: Valery Kholodkov
Excel in Nginx quickly by learning to use its most essential features in real-life applications.
This short review comes from this book or the store.
Authors: Derek DeJonghe
You’ll find recipes for:
This short review comes from this book or the store.
Authors: Martin Fjordvald, Clement Nedelcu
Harness the power of Nginx to make the most of your infrastructure and serve pages faster than ever.
This short review comes from this book or the store.
Authors: Rahul Sharma
Optimize NGINX for high-performance, scalable web applications.
This short review comes from this book or the store.
Authors: Dimitri Aivaliotis
Written for experienced systems administrators and engineers, this book teaches you from scratch how to configure Nginx for any situation. Step-by-step instructions and real-world code snippets clarify even the most complex areas.
This short review comes from this book or the store.
Authors: Faisal Memon, Owen Garrett, Michael Pleshakov
Learn in this ebook how to get started with ModSecurity, the world’s most widely deployed web application firewall (WAF), now available for NGINX and NGINX Plus.
This short review comes from this book or the store.
Authors: Faisal Memon
This ebook provides step-by-step instructions on replacing Cisco ACE with NGINX and off-the-shelf servers. NGINX helps you cut costs and modernize.
In this ebook you will learn:
This short review comes from this book or the store.
:black_small_square: Nginx Project
:black_small_square: Nginx Documentation
:black_small_square: Nginx Wiki
:black_small_square: Nginx Admin's Guide
:black_small_square: Nginx Pitfalls and Common Mistakes
:black_small_square: Nginx Forum
:black_small_square: Nginx Mailing List
:black_small_square: Nginx Read-only Mirror
:black_small_square: OpenResty
:black_small_square: The Tengine Web Server
:black_small_square: NGINX vs. Apache (Pro/Con Review, Uses, & Hosting for Each)
:black_small_square: Web cache server performance benchmark: nuster vs nginx vs varnish vs squid
:black_small_square: Nginx Cheatsheet
:black_small_square: Nginx Tutorials, Linux Sysadmin Configuration & Optimizing Tips and Tricks
:black_small_square: Nginx boilerplate configs
:black_small_square: Awesome Nginx configuration template
:black_small_square: Nginx Quick Reference
:black_small_square: A collection of resources covering Nginx and more
:black_small_square: A collection of useful Nginx configuration snippets
:black_small_square: agentzh's Nginx Tutorials
:black_small_square: Nginx Tuning For Best Performance by Denji
:black_small_square: TLS has exactly one performance problem: it is not used widely enough
:black_small_square: SSL/TLS Deployment Best Practices
:black_small_square: SSL Server Rating Guide
:black_small_square: How to Build a Tough NGINX Server in 15 Steps
:black_small_square: Top 25 Nginx Web Server Best Security Practices
:black_small_square: Nginx Secure Web Server
:black_small_square: Strong ciphers for Apache, Nginx, Lighttpd and more
:black_small_square: Strong SSL Security on Nginx
:black_small_square: Enable cross-origin resource sharing (CORS)
:black_small_square: NAXSI - WAF for Nginx
:black_small_square: ModSecurity for Nginx
:black_small_square: Transport Layer Protection Cheat Sheet
:black_small_square: Security/Server Side TLS
:black_small_square: NGINX Rate Limit, Burst and nodelay sandbox
:black_small_square: Nginx config generator on steroids
:black_small_square: Mozilla SSL Configuration Generator
:black_small_square: gixy - is a tool to analyze Nginx configuration to prevent security misconfiguration and automate flaw detection.
:black_small_square: nginx-config-formatter - Nginx config file formatter/beautifier written in Python.
:black_small_square: nginxbeautifier - format and beautify nginx config files.
:black_small_square: nginx-minify-conf - creates a minified version of a nginx configuration.
:black_small_square: GoAccess - is a fast, terminal-based log analyzer (quickly analyze and view web server statistics in real time).
:black_small_square: Graylog - is a leading centralized log management for capturing, storing, and enabling real-time analysis.
:black_small_square: Logstash - is an open source, server-side data processing pipeline.
:black_small_square: ngxtop - parses your nginx access log and outputs useful, top-like, metrics of your nginx server.
:black_small_square: ab - is a single-threaded command line tool for measuring the performance of HTTP web servers.
:black_small_square: siege - is an http load testing and benchmarking utility.
:black_small_square: wrk - is a modern HTTP benchmarking tool capable of generating significant load.
:black_small_square: bombardier - is a HTTP(S) benchmarking tool.
:black_small_square: gobench - is a HTTP/HTTPS load testing and benchmarking tool.
:black_small_square: hey - is a HTTP load generator, ApacheBench (ab) replacement, formerly known as rakyll/boom.
:black_small_square: boom - is a script you can use to quickly smoke-test your web app deployment.
:black_small_square: JMeter™ - is designed to load test functional behavior and measure performance.
:black_small_square: Gatling - is a powerful open-source load and performance testing tool for web applications.
:black_small_square: locust - is an easy-to-use, distributed, user load testing tool.
:black_small_square: htrace.sh - is a simple Swiss Army knife for http/https troubleshooting and profiling.
:black_small_square: Emiller’s Guide To Nginx Module Development
:black_small_square: An Introduction To OpenResty (nginx + lua) - Part 1
:black_small_square: An Introduction To OpenResty - Part 2 - Concepts
:black_small_square: An Introduction To OpenResty - Part 3
:black_small_square: SSL Server Test by SSL Labs
:black_small_square: SSL/TLS Capabilities of Your Browser
:black_small_square: Test SSL/TLS (PCI DSS, HIPAA and NIST)
:black_small_square: SSL analyzer and certificate checker
:black_small_square: Test your TLS server configuration (e.g. ciphers)
:black_small_square: Scan your website for non-secure content
:black_small_square: Public Diffie-Hellman Parameter Service/Tool
:black_small_square: Analyse the HTTP response headers by Security Headers
:black_small_square: Analyze your website by Mozilla Observatory
:black_small_square: CAA Record Helper
:black_small_square: Linting tool that will help you with your site's accessibility, speed, security and more
:black_small_square: Service to scan and analyse websites
:black_small_square: Tool from above to either encode or decode a string of text
:black_small_square: Online translator for search queries on log data
:black_small_square: Online regex tester and debugger: PHP, PCRE, Python, Golang and JavaScript
:black_small_square: Online tool to learn, build, & test Regular Expressions
:black_small_square: Online Regex Tester & Debugger
:black_small_square: A web app for encryption, encoding, compression and data analysis
:black_small_square: Nginx location match tester
:black_small_square: Nginx location match visible
:black_small_square: Transport Layer Security (TLS) Parameters
:black_small_square: Security/Server Side TLS by Mozilla
:black_small_square: TLS Security 6: Examples of TLS Vulnerabilities and Attacks
:black_small_square: Guidelines for Setting Security Headers
:black_small_square: Secure your web application with these HTTP headers
:black_small_square: Security HTTP Headers
:black_small_square: Analysis of various reverse proxies, cache proxies, load balancers, etc.
:black_small_square: TLS Redirection (and Virtual Host Confusion)
:black_small_square: HTTPS on Stack Overflow: The End of a Long Road
:black_small_square: The Architecture of Open Source Applications - Nginx
:black_small_square: BBC Digital Media Distribution: How we improved throughput by 4x
:black_small_square: The C10K problem by Dan Kegel
:black_small_square: High Performance Browser Networking
:black_small_square: jemalloc vs tcmalloc vs dlmalloc
If you compile NGINX server by default all files and directories are available from
/usr/local/nginx
location.
For prebuilt NGINX package paths can be as follows:
/etc/nginx
- is the default configuration root for the NGINX service/usr/local/etc/nginx
, /usr/local/nginx/conf
/etc/nginx/nginx.conf
- is the default configuration entry point used by the NGINX services, includes the top-level http block and all other configuration contexts and files/usr/local/etc/nginx/nginx.conf
, /usr/local/nginx/conf/nginx.conf
/usr/share/nginx
- is the default root directory for requests, contains html
directory and basic static files/var/log/nginx
- is the default log (access and error log) location for NGINX/var/cache/nginx
- is the default temporary files location for NGINX/var/lib/nginx
/etc/nginx/conf
- contains custom/vhosts configuration files/etc/nginx/conf.d
, /etc/nginx/sites-enabled
/var/run/nginx
- contains information about NGINX process(es)/usr/local/nginx/logs
nginx -h
- shows the helpnginx -v
- shows the NGINX versionnginx -V
- shows the extended information about NGINX: version, build parameters and configuration argumentsnginx -t
- tests the NGINX configurationnginx -c
- sets configuration file (default: /etc/nginx/nginx.conf
)nginx -p
- sets prefix path (default: /etc/nginx/
)nginx -T
- tests the NGINX configuration and prints the validated configuration on the screennginx -s
- sends a signal to the NGINX master process:
stop
- discontinues the NGINX process immediatelyquit
- stops the NGINX process after it finishes processing inflight requestsreload
- reloads the configuration without stopping NGINX processesreopen
- instructs NGINX to reopen log filesnginx -g
- sets global directives out of configuration fileNGINX configuration files don't support comment blocks; they only accept #
at the beginning of a line for a comment.
Lines containing directives must end with a ;
or NGINX will fail to load the configuration and report an error.
Variables in NGINX start with $
. Some modules introduce variables can be used when setting directives.
There are some directives that do not support variables, e.g.
access_log
orerror_log
.
Strings may be inputted without quotes unless they include blank spaces, semicolons or curly braces, then they need to be escaped with backslashes or enclosed in single/double quotes.
Variables in quoted strings are expanded normally unless the $
is escaped.
Read this great article about the NGINX configuration inheritance model by Martin Fjordvald.
Configuration options are called directives. We have four types of directives in NGINX:
worker_connections 512;
error_log /var/log/nginx/localhost/localhost-error.log warn;
rewrite ^(.*)$ /msie/$1 break;
try_files
directive:try_files $uri $uri/ /test/index.html;
Directives are organized into groups known as blocks or contexts. Generally context is a block directive can have other directives inside braces. It appears to be organized in a tree-like structure, defined by sets of brackets - {
and }
. It's a simple structure and very transparent.
As a general rule, if a directive is valid in multiple nested scopes, a declaration in a broader context will be passed on to any child contexts as default values.
Directives placed in the configuration file outside of any contexts are considered to be in the global/main context.
Contexts can be layered within one another so NGINX provides a level of inheritance. Their structure looks like this:
Global/Main Context
|
|
+-----» Events Context
|
|
+-----» HTTP Context
| |
| |
| +-----» Server Context
| | |
| | |
| | +-----» Location Context
| |
| |
| +-----» Upstream Context
|
|
+-----» Mail Context
Sizes can be specified in:
k
or K
: Kilobytesm
or M
: Megabytesg
or G
: Gigabytesclient_max_body_size 2M;
Time intervals can be specified in:
ms
: Millisecondss
: Seconds (default, without a suffix)m
: Minutesh
: Hoursd
: Daysw
: WeeksM
: Months (30 days)y
: Years (365 days)proxy_read_timeout 20s;
# 1) Download vim plugin for NGINX:
# Official NGINX vim plugin:
mkdir -p ~/.vim/syntax/
wget "http://www.vim.org/scripts/download_script.php?src_id=19394" -O ~/.vim/syntax/nginx.vim
# Improved NGINX vim plugin (incl. syntax highlighting) with Pathogen:
mkdir -p ~/.vim/{autoload,bundle}/
curl -LSso ~/.vim/autoload/pathogen.vim https://tpo.pe/pathogen.vim
echo -en "\nexecute pathogen#infect()\n" >> ~/.vimrc
git clone https://github.com/chr4/nginx.vim ~/.vim/bundle/nginx.vim
# 2) Set location of NGINX config files:
cat > ~/.vim/filetype.vim << __EOF__
au BufRead,BufNewFile /etc/nginx/*,/etc/nginx/conf.d/*,/usr/local/nginx/conf/*,*/conf/nginx.conf if &ft == '' | setfiletype nginx | endif
__EOF__
Install cabal
- system for building and packaging Haskell libraries and programs (on Ubuntu):
add-apt-repository -y ppa:hvr/ghc
apt-get update
apt-get install -y cabal-install-1.22 ghc-7.10.2
# Add this to your shell main configuration file:
export PATH=$HOME/.cabal/bin:/opt/cabal/1.22/bin:/opt/ghc/7.10.2/bin:$PATH
source $HOME/.<shellrc>
cabal update
nginx-lint
:
git clone https://github.com/temoto/nginx-lint
cd nginx-lint && cabal install --global
sublime-nginx
+ SublimeLinter-contrib-nginx-lint
:
Bring up the Command Palette and type install
. Among the commands you should see Package Control: Install Package. Type nginx
to install sublime-nginx and after that do the above again for install SublimeLinter-contrib-nginx-lint: type SublimeLinter-contrib-nginx-lint
.
NGINX has one master process and one or more worker processes.
The main purposes of the master process is to read and evaluate configuration files, as well as maintain the worker processes (respawn when a worker dies), handle signals, notify workers, opens log files, and, of course binding to ports.
Master process should be started as root user, because this will allow NGINX to open sockets below 1024 (it needs to be able to listen on port 80 for HTTP and 443 for HTTPS).
The worker processes do the actual processing of requests and get commands from master process. They runs in an event loop, handle network connections, read and write content to disk, and communicate with upstream servers. These are spawned by the master process, and the user and group will as specified (unprivileged).
NGINX has also cache loader and cache manager processes but only if you enable caching.
The following signals can be sent to the NGINX master process:
SIGNAL | DESCRIPTION |
---|---|
TERM , INT |
quick shutdown |
QUIT |
graceful shutdown |
KILL |
halts a stubborn process |
HUP |
configuration reload, start new workers, gracefully shutdown the old worker processes |
USR1 |
reopen the log files |
USR2 |
upgrade executable on the fly |
WINCH |
gracefully shutdown the worker processes |
There’s no need to control the worker processes yourself. However, they support some signals too:
SIGNAL | DESCRIPTION |
---|---|
TERM , INT |
quick shutdown |
QUIT |
graceful shutdown |
USR1 |
reopen the log files |
NGINX supports a variety of connection processing methods which depends on the platform used. For more information please see connection processing methods explanation.
In general there are four types of event multiplexing:
select
- is anachronism and not recommended but installed on all platforms as a fallbackpoll
- is anachronism and not recommendedepoll
- recommend if you're using GNU/Linuxkqueue
- recommend if you're using BSD (is technically superior to epoll
)There are also great resources (and comparisons) about them:
Look also at this libevent benchmark:
This infographic comes from daemonforums - An interesting benchmark (kqueue vs. epoll).
You may also view why big players use NGINX on FreeBSD instead of on GNU/Linux:
Okay, so how many simultaneous connections can be processed by NGINX?
worker_processes * worker_connections = max clients (in theory)
According to this: if you are only running 2 worker processes with 512 worker connections, you will be able to serve 1024 clients.
It is a bit confusing because the sum of
worker_processes
andworker_connections
does not directly translate into the number of clients that can be served simultaneously. Each clients (e.g. browsers) opens a number of parallel connections to download various components that compose a web page, for example, images, scripts, and so on.
NGX_HTTP_POST_READ_PHASE
- first phase, read the request header
NGX_HTTP_SERVER_REWRITE_PHASE
- implementation of rewrite directives defined in a server block
NGX_HTTP_FIND_CONFIG_PHASE
- replace the location according to URI (location lookup)NGX_HTTP_REWRITE_PHASE
- URI transformation on location level
NGX_HTTP_POST_REWRITE_PHASE
- URI transformation post-processing (the request is redirected to a new location)
NGX_HTTP_PREACCESS_PHASE
- authentication preprocessing request limit, connection limit (access restriction)
NGX_HTTP_ACCESS_PHASE
- verification of the client (the authentication process, limiting access)
NGX_HTTP_POST_ACCESS_PHASE
- access restrictions check post-processing phase, the certification process, processing satisfy any
directive
NGX_HTTP_PRECONTENT_PHASE
- generating content
NGX_HTTP_CONTENT_PHASE
- content processing
NGX_HTTP_LOG_PHASE
- log processing
You may feel lost now (me too...) so I let myself put this great preview:
This infographic comes from Inside NGINX official library.
NGINX does have server blocks (like a virtual hosts is an Apache) that use
listen
andserver_name
directives to bind to tcp sockets.
Before start reading this chapter you should know what regular expressions are and how they works. I recommend two great and short write-ups about regular expressions created by Jonny Fox:
Why? Regular expressions can be used in both the server_name
and location
directives, and sometimes you must have a great skill of reading them. I think you should create the most readable regular expressions that do not become spaghetti code - impossible to debug and maintain.
It's short example of server block context (two server blocks):
http {
index index.html;
root /var/www/example.com/default;
server {
listen 10.10.250.10:80;
server_name www.example.com;
access_log logs/example.access.log main;
root /var/www/example.com/public;
...
}
server {
listen 10.10.250.11:80;
server_name "~^(api.)?example\.com api.de.example.com";
access_log logs/example.access.log main;
proxy_pass http://localhost:8080;
...
}
}
NGINX uses the following logic to determining which virtual server (server block) should be used:
Match the address:port
pair to the listen
directive - that can be multiple server blocks with listen
directives of the same specificity that can handle the request
NGINX use the
address:port
combination for handle incoming connections. This pair is assigned to thelisten
directive.
The listen
directive can be set to:
127.0.0.1:80;
)80
is used (127.0.0.1;
) - becomes 127.0.0.1:80;
80;
or *:80;
) - becomes 0.0.0.0:80;
unix:/var/run/nginx.sock;
)If the listen
directive is not present then either *:80
is used (runs with the superuser privileges), or *:8000
otherwise.
The next steps are as follows:
NGINX translates all incomplete listen
directives by substituting missing values with their default values (see above)
NGINX attempts to collect a list of the server blocks that match the request most specifically based on the address:port
If any block that is functionally using 0.0.0.0
, will not be selected if there are matching blocks that list a specific IP
If there is only one most specific match, that server block will be used to serve the request
If there are multiple server blocks with the same level of matching, NGINX then begins to evaluate the server_name
directive of each server block
Look at this short example:
# From client:
GET / HTTP/1.0
Host: api.random.com
server {
# This block will be processed:
listen 192.168.252.10; # --> 192.168.252.10:80
...
}
server {
listen 80; # --> *:80 --> 0.0.0.0:80
server_name api.random.com;
...
}
Match the Host
header field against the server_name
directive as a string (the exact names hash table)
Match the Host
header field against the server_name
directive with a wildcard at the beginning of the string (the hash table with wildcard names starting with an asterisk)
If one is found, that block will be used to serve the request. If multiple matches are found, the longest match will be used to serve the request.
Match the Host
header field against the server_name
directive with a wildcard at the end of the string (the hash table with wildcard names ending with an asterisk)
If one is found, that block is used to serve the request. If multiple matches are found, the longest match will be used to serve the request.
Match the Host
header field against the server_name
directive as a regular expression
The first
server_name
with a regular expression that matches theHost
header will be used to serve the request.
If all the Host
headers doesn't match, then direct to the listen
directive marked as default_server
If all the Host
headers doesn't match and there is no default_server
, direct to the first server with a listen
directive that satisfies first step
Finally, NGINX goes to the location
context
This short list is based on Mastering Nginx - The virtual server section.
For each request, NGINX goes through a process to choose the best location block that will be used to serve that request.
Let's short introduction something about this:
^~
and (none)
, if this match used the ^~
prefix, searching stops~
and ~*
; in the order they are defined in the configuration fileShort example from the Nginx documentation:
location = / {
# Matches the query / only.
[ configuration A ]
}
location / {
# Matches any query, since all queries begin with /, but regular
# expressions and any longer conventional blocks will be
# matched first.
[ configuration B ]
}
location /documents/ {
# Matches any query beginning with /documents/ and continues searching,
# so regular expressions will be checked. This will be matched only if
# regular expressions don't find a match.
[ configuration C ]
}
location ^~ /images/ {
# Matches any query beginning with /images/ and halts searching,
# so regular expressions will not be checked.
[ configuration D ]
}
location ~* \.(gif|jpg|jpeg)$ {
# Matches any request ending in gif, jpg, or jpeg. However, all
# requests to the /images/ directory will be handled by
# Configuration D.
[ configuration E ]
}
To help you understand how does location match works:
The location syntax looks like:
location optional_modifier location_match { ... }
location_match
in the above defines what NGINX should check the request URI against. The optional_modifier
below will cause the associated location block to be interpreted as follows:
(none)
: if no modifiers are present, the location is interpreted as a prefix match. To determine a match, the location will now be matched against the beginning of the URI
=
: is an exact match, without any wildcards, prefix matching or regular expressions; forces a literal match between the request URI and the location parameter
~
: if a tilde modifier is present, this location must be used for case sensitive matching (RE match)
~*
: if a tilde and asterisk modifier is used, the location must be used for case insensitive matching (RE match)
^~
: assuming this block is the best non-RE match, a carat followed by a tilde modifier means that RE matching will not take place
The process of choosing NGINX location block is as follows:
Prefix-based NGINX location matches (no regular expression). Each location will be checked against the request URI
NGINX searches for an exact match. If a "=" modifier exactly matches the request URI, this specific location block is chosen right away
If no exact (meaning no "=" modifier) location block is found, NGINX will continue with non-exact prefixes. It starts with the longest matching prefix location for this URI, with the following approach:
In case the longest matching prefix location has the "^~" modifier, NGINX will stop its search right away and choose this location
Assuming the longest matching prefix location doesn’t use the "^~"modifier, the match is temporarily stored and the process continues
As soon as the longest matching prefix location is chosen and stored, NGINX continues to evaluate the case-sensitive and insensitive regular expression locations. The first regular expression location that fits the URI is selected right away to process the request
If no regular expression locations are found that match the request URI, the previously stored prefix location is selected to serve the request
In order to better understand how this process work please see this short cheatsheet that will allow you to design your location blocks in a predictable way:
I recommend to use external tools for testing regular expressions. For more please see online tools chapter.
Ok, so we have following more complicated configuration:
server {
listen 80;
server_name xyz.com www.xyz.com;
location ~ ^/(media|static)/ {
root /var/www/xyz.com/static;
expires 10d;
}
location ~* ^/(media2|static2) {
root /var/www/xyz.com/static2;
expires 20d;
}
location /static3 {
root /var/www/xyz.com/static3;
}
location ^~ /static4 {
root /var/www/xyz.com/static4;
}
location = /api {
proxy_pass http://127.0.0.1:8080;
}
location / {
proxy_pass http://127.0.0.1:8080;
}
location /backend {
proxy_pass http://127.0.0.1:8080;
}
location ~ logo.xcf$ {
root /var/www/logo;
expires 48h;
}
location ~* .(png|ico|gif|xcf)$ {
root /var/www/img;
expires 24h;
}
location ~ logo.ico$ {
root /var/www/logo;
expires 96h;
}
location ~ logo.jpg$ {
root /var/www/logo;
expires 48h;
}
}
And here is the table with the results:
URL | LOCATIONS FOUND | FINAL MATCH |
---|---|---|
/ |
1) prefix match for / |
/ |
/css |
1) prefix match for / |
/ |
/api |
1) exact match for /api |
/api |
/api/ |
1) prefix match for / |
/ |
/backend |
1) prefix match for / 2) prefix match for /backend |
/backend |
/static |
1) prefix match for / |
/ |
/static/header.png |
1) prefix match for / 2) case sensitive regex match for ^/(media\|static)/ |
^/(media\|static)/ |
/static/logo.jpg |
1) prefix match for / 2) case sensitive regex match for ^/(media\|static)/ |
^/(media\|static)/ |
/media2 |
1) prefix match for / 2) case insensitive regex match for ^/(media2\|static2) |
^/(media2\|static2) |
/media2/ |
1) prefix match for / 2) case insensitive regex match for ^/(media2\|static2) |
^/(media2\|static2) |
/static2/logo.jpg |
1) prefix match for / 2) case insensitive regex match for ^/(media2\|static2) |
^/(media2\|static2) |
/static2/logo.png |
1) prefix match for / 2) case insensitive regex match for ^/(media2\|static2) |
^/(media2\|static2) |
/static3/logo.jpg |
1) prefix match for /static3 2) prefix match for / 3) case sensitive regex match for logo.jpg$ |
logo.jpg$ |
/static3/logo.png |
1) prefix match for /static3 2) prefix match for / 3) case insensitive regex match for .(png\|ico\|gif\|xcf)$ |
.(png\|ico\|gif\|xcf)$ |
/static4/logo.jpg |
1) priority prefix match for /static4 2) prefix match for / |
/static4 |
/static4/logo.png |
1) priority prefix match for /static4 2) prefix match for / |
/static4 |
/static5/logo.jpg |
1) prefix match for / 2) case sensitive regex match for logo.jpg$ |
logo.jpg$ |
/static5/logo.png |
1) prefix match for / 2) case insensitive regex match for .(png\|ico\|gif\|xcf)$ |
.(png\|ico\|gif\|xcf)$ |
/static5/logo.xcf |
1) prefix match for / 2) case sensitive regex match for logo.xcf$ |
logo.xcf$ |
/static5/logo.ico |
1) prefix match for / 2) case insensitive regex match for .(png\|ico\|gif\|xcf)$ |
.(png\|ico\|gif\|xcf)$ |
The following is a list of all severity levels:
TYPE | DESCRIPTION |
---|---|
debug |
information that can be useful to pinpoint where a problem is occurring |
info |
informational messages that aren’t necessary to read but may be good to know |
notice |
something normal happened that is worth noting |
warn |
something unexpected happened, however is not a cause for concern |
error |
something was unsuccessful, contains the action of limiting rules |
crit |
important problems that need to be addressed |
alert |
severe situation where action is needed promptly |
emerg |
the system is in an unusable state and requires immediate attention |
For example: if you set crit
error log level, messages of crit
, alert
, and emerg
levels are logged.
All rate limiting rules (definitions) should be added to the NGINX
http
context.
Rate limiting rules are useful for:
NGINX has following variables (unique keys) that can be used in a rate limiting rules:
VARIABLE | DESCRIPTION |
---|---|
$remote_addr |
client address |
$binary_remote_addr |
client address in a binary form, it is smaller and saves space then remote_addr |
$server_name |
name of the server which accepted a request |
$request_uri |
full original request URI (with arguments) |
$query_string |
arguments in the request line |
Please see official doc for more information about variables.
NGINX also provides following keys:
KEY | DESCRIPTION |
---|---|
limit_req_zone |
stores the current number of excessive requests |
limit_conn_zone |
stores the maximum allowed number of connections |
and directives:
DIRECTIVE | DESCRIPTION |
---|---|
limit_req |
sets the shared memory zone and the maximum burst size of requests |
limit_conn |
sets the shared memory zone and the maximum allowed number of connections to the server per a client IP |
Keys are used to store the state of each IP address and how often it has accessed a limited object. This information are stored in shared memory available from all NGINX worker processes.
Both keys also provides response status parameters indicating too many requests or connections with specific http code (default 503).
limit_req_status <value>
limit_conn_status <value>
For example, if you want to set the desired logging level for cases when the server limits the number of connections:
# Add this to http context:
limit_req_status 429;
# Set your own error page for 429 http code:
error_page 429 /rate_limit.html;
location = /rate_limit.html {
root /usr/share/www/http-error-pages/sites/other;
internal;
}
# And create this file:
cat > /usr/share/www/http-error-pages/sites/other/rate_limit.html << __EOF__
HTTP 429 Too Many Requests
__EOF__
Rate limiting rules also have zones that lets you define a shared space in which to count the incoming requests or connections.
All requests or connections coming into the same space will be counted in the same rate limit. This is what allows you to limit per URL, per IP, or anything else.
The zone has two required parts:
<name>
- is the zone identifier<size>
- is the zone sizeExample:
<key> <variable> zone=<name>:<size>;
State information for about 16,000 IP addresses takes 1 megabyte. So 1 kilobyte zone has 16 IP addresses.
The range of zones is as follows:
http {
... zone=<name>;
server {
... zone=<name>;
location /api {
... zone=<name>;
limit_req_zone
key lets you set rate
parameter (optional) - it defines the rate limited URL(s).
For enable queue you should use limit_req
or limit_conn
directives (see above). limit_req
also provides optional parameters:
PARAMETER | DESCRIPTION |
---|---|
burst=<num> |
sets the maximum number of excessive requests that await to be processed in a timely manner; maximum requests as rate * burst in burst seconds |
nodelay |
it imposes a rate limit without constraining the allowed spacing between requests; default NGINX would return 503 response and not handle excessive requests |
nodelay
parameters are only useful when you also set aburst
.
Without nodelay
option NGINX would wait (no 503 response) and handle excessive requests with some delay.
It is an essential way for testing NGINX configuration:
nginx -t -c /etc/nginx/nginx.conf;
An external tool for analyse NGINX configuration is gixy
:
gixy /etc/nginx/nginx.conf
Paths configuration file:
/etc/goaccess.conf
/etc/goaccess/goaccess.conf
/usr/local/etc/goaccess.conf
Prior to start GoAccess enable these parameters:
time-format %H:%M:%S
date-format %d/%b/%Y
log-format %h %^[%d:%t %^] "%r" %s %b "%R" "%u"
# Ubuntu/Debian
apt-get install gcc make libncursesw5-dev libgeoip-dev libtokyocabinet-dev
# RHEL/CentOS
yum install gcc ncurses-devel geoip-devel tokyocabinet-devel
cd /usr/local/src/
wget -c https://tar.goaccess.io/goaccess-1.3.tar.gz && \
tar xzvfp goaccess-1.3.tar.gz
cd goaccess-1.3
./configure --enable-utf8 --enable-geoip=legacy --with-openssl=<path_to_openssl_sources> --sysconfdir=/etc/
make -j2 && make install
ln -s /usr/local/bin/goaccess /usr/bin/goaccess
Default path to configuration file:
/etc/goaccess/goaccess.conf
. You can always copy it from/usr/local/src/goaccess-<version>/config/goaccess.conf
source tree.
goaccess -f access.log -a
zcat access.log.1.gz | goaccess -a -p /etc/goaccess/goaccess.conf
ssh user@remote_host 'access.log' | goaccess -a
goaccess -p /etc/goaccess/goaccess.conf -f access.log --log-format=COMBINED -o /var/www/index.html
ngxtop -l access.log
ngxtop -l access.log -i 'status >= 400' print request status
ssh user@remote_host tail -f access.log | ngxtop -f combined
You can change combinations and parameters of these commands.
# 1)
curl -Iks <scheme>://<server_name>:<port>
# 2)
http -p Hh <scheme>://<server_name>:<port>
# 3)
htrace.sh -u <scheme>://<server_name>:<port> -h
# 1)
curl -Iks --location -X GET -A "x-agent" <scheme>://<server_name>:<port>
# 2)
http -p Hh GET <scheme>://<server_name>:<port> User-Agent:x-agent --follow
# 3)
htrace.sh -u <scheme>://<server_name>:<port> -M GET --user-agent "x-agent" -h
# URL sequence substitution with a dummy query string:
curl -ks <scheme>://<server_name>:<port>?[1-20]
# With shell 'for' loop:
for i in {1..20} ; do curl -ks <scheme>://<server_name>:<port> ; done
# 1)
echo | openssl s_client -connect <server_name>:<port>
# 2)
gnutls-cli --disable-sni -p 443 <server_name>
# 1)
echo | openssl s_client -servername <server_name> -connect <server_name>:<port>
# 2)
gnutls-cli -p 443 <server_name>
openssl s_client -tls1_2 -connect <server_name>:<port>
openssl s_client -cipher 'AES128-SHA' -connect <server_name>:<port>
Only use this for testing the availability, performance and capacity planning of a web application of your environment.
hping3 -V -c 1000000 -d 120 -S -w 64 -p 80 --flood --rand-source <remote_host>
Only use this for testing the availability, performance and capacity planning of a web application of your environment.
# 1)
slowhttptest -g -o http_dos.stats -H -c 1000 -i 15 -r 200 -t GET -x 24 -p 3 -u <scheme>://<server_name>/index.php
slowhttptest -g -o http_dos.stats -B -c 5000 -i 5 -r 200 -t POST -l 180 -x 5 -u <scheme>://<server_name>/service/login
# 2)
pip3 install slowloris
slowloris <server_name>
# 3)
git clone https://github.com/jseidl/GoldenEye && cd GoldenEye
./goldeneye.py <scheme>://<server_name> -w 150 -s 75 -m GET
You can change combinations and parameters of these commands. When carrying out the analysis, remember about debug log and log formats.
with ps
:
ps axw -o pid,ppid,gid,user,etime,%cpu,%mem,vsz,rss,wchan,ni,command | egrep '([n]ginx|[P]ID)'
with top
:
top -p $(pgrep -d , nginx)
nginx -V 2>&1 | grep -- 'http_geoip_module'
# - add `head -n X` to the end to limit the result
# - add this to the end for print header:
# ... | xargs printf '%10s s\n%10s s\n' "AMOUNT" "IP_ADDRESS"
awk '{print $1}' access.log | sort | uniq -c | sort -nr
# - add this to the end for print header:
# ... | xargs printf '%10s%10s s\n%10s%10s s\n' "NUM" "AMOUNT" "IP_ADDRESS"
cut -d ' ' -f1 access.log | sort | uniq -c | sort -nr | head -5 | nl
# - add `head -n X` to the end to limit the result
# - add this to the end for print header:
# ... | xargs printf '%10s\t%s\n%10s\t%s\n' "AMOUNT" "URL"
awk -F\" '{print $2}' access.log | awk '{print $2}' | sort | uniq -c | sort -nr
# - add `head -n X` to the end to limit the result
# - add this to the end for print header:
# ... | xargs printf '%10s\t%s\n%10s\t%s\n' "AMOUNT" "URL"
awk -F\" '($2 ~ "../../string") { print $2}' access.log | awk '{print $2}' | sort | uniq -c | sort -nr
# - add `head -n X` to the end to limit the result
# - add this to the end for print header:
# ... | xargs printf '%10s %8s\t%s\n%10s %8s\t%s\n' "AMOUNT" "METHOD" "URL"
awk -F\" '{print $2}' access.log | awk '{print $1 "\t" $2}' | sort | uniq -c | sort -nr
# - add `head -n X` to the end to limit the result
# - add this to the end for print header:
# ... | xargs printf '%10s\t%s\n%10s\t%s\n' "AMOUNT" "HTTP_CODE"
awk '{print $9}' access.log | sort | uniq -c | sort -nr
tail -n 100 -f access.log | grep "HTTP/[1-2].[0-1]\" [2]"
tail -n 100 -f access.log | grep "HTTP/[1-2].[0-1]\" [5]"
# - add `head -n X` to the end to limit the result
# - add this to the end for print header:
# ... | xargs printf '%10s\t%s\n%10s\t%s\n' "AMOUNT" "URL"
awk '($9 ~ /502/)' access.log | awk '{print $7}' | sort | uniq -c | sort -nr
# - add `head -n X` to the end to limit the result
# - add this to the end for print header:
# ... | xargs printf '%10s\t%s\n%10s\t%s\n' "AMOUNT" "URL"
awk '($9 ~ /401/)' access.log | awk -F\" '($2 ~ "../../*.php")' | awk '{print $7}' | sort | uniq -c | sort -nr
# Not less than 1 minute:
tail -2000 access.log | awk -v date=$(date -d '1 minutes ago' +"%d/%b/%Y:%H:%M") '$4 ~ date' | cut -d '"' -f3 | cut -d ' ' -f2 | sort | uniq -c | sort -nr
# Last 2000 requests from log file:
# - add this to the end for print header:
# ... | xargs printf '%10s\t%s\n%10s\t%s\n' "AMOUNT" "HTTP_CODE"
tail -2000 access.log | cut -d '"' -f3 | cut -d ' ' -f2 | sort | uniq -c | sort -nr
# In real time:
tail -F access.log | pv -lr >/dev/null
# - add `head -n X` to the end to limit the result
# - add this to the end for print header:
# ... | xargs printf '%10s%24s%18s\n%10s%24s%18s\n' "AMOUNT" "DATE" "IP_ADDRESS"
awk '{print $4 " " $1}' access.log | uniq -c | sort -nr | tr -d "["
# - add `head -n X` to the end to limit the result
# - add this to the end for print header:
# ... | xargs printf '%10s%24s%18s\n%10s%24s%18s\n' "AMOUNT" "DATE" "IP_ADDRESS"
awk '{print $4 " " $1}' access.log | uniq -c | sort -nr | tr -d "["
# - add `head -n X` to the end to limit the result
# - add this to the end for print header:
# ... | xargs printf '%10s%24s%18s\t%s\n%10s%24s%18s\t%s\n' "AMOUNT" "DATE" "IP_ADDRESS" "URL"
awk '{print $4 " " $1 " " $7}' access.log | uniq -c | sort -nr | tr -d "["
awk -v _date=`date -d 'now-6 hours' +[%d/%b/%Y:%H:%M:%S` ' { if ($4 > _date) print $0}' access.log
# date command shows output for specific locale, for prevent this you should set LANG variable:
awk -v _date=$(LANG=en_us.utf-8 date -d 'now-6 hours' +[%d/%b/%Y:%H:%M:%S) ' { if ($4 > _date) print $0}' access.log
# or:
export LANG=en_us.utf-8
awk -v _date=$(date -d 'now-6 hours' +[%d/%b/%Y:%H:%M:%S) ' { if ($4 > _date) print $0}' access.log
# 1)
awk '$4>"[05/Feb/2019:02:10" && $4<"[15/Feb/2019:08:20"' access.log
# 2)
# date command shows output for specific locale, for prevent this you should set LANG variable:
awk -v _dateB=$(LANG=en_us.utf-8 date -d '10:20' +[%d/%b/%Y:%H:%M:%S) -v _dateE=$(LANG=en_us.utf-8 date -d '20:30' +[%d/%b/%Y:%H:%M:%S) ' { if ($4 > _dateB && $4 < _dateE) print $0}' access.log
# or:
export LANG=en_us.utf-8
awk -v _dateB=$(date -d '10:20' +[%d/%b/%Y:%H:%M:%S) -v _dateE=$(date -d '20:30' +[%d/%b/%Y:%H:%M:%S) ' { if ($4 > _dateB && $4 < _dateE) print $0}' access.log
# 3)
# date command shows output for specific locale, for prevent this you should set LANG variable:
awk -v _dateB=$(LANG=en_us.utf-8 date -d 'now-12 hours' +[%d/%b/%Y:%H:%M:%S) -v _dateE=$(LANG=en_us.utf-8 date -d 'now-2 hours' +[%d/%b/%Y:%H:%M:%S) ' { if ($4 > _dateB && $4 < _dateE) print $0}' access.log
# or:
export LANG=en_us.utf-8
awk -v _dateB=$(date -d 'now-12 hours' +[%d/%b/%Y:%H:%M:%S) -v _dateE=$(date -d 'now-2 hours' +[%d/%b/%Y:%H:%M:%S) ' { if ($4 > _dateB && $4 < _dateE) print $0}' access.log
tail -F access.log | pv -N RAW -lc 1>/dev/null
strace -e trace=network -p `pidof nginx | sed -e 's/ /,/g'`
strace -ff -e trace=file nginx 2>&1 | perl -ne 's/^[^"]+"(([^\\"]|\\[\\"nt])*)".*/$1/ && print'
gzip_static
module is workingstrace -p `pidof nginx | sed -e 's/ /,/g'` 2>&1 | grep gz
Example 1 (more elegant way):
log_format debug-req-trace
'$pid - "$request_method $scheme://$host$request_uri" '
'$remote_addr:$remote_port $server_addr:$server_port '
'$request_id';
# Output example:
31863 - "GET https://example.com/" 35.228.233.xxx:63784 10.240.20.2:443 be90154db5beb0e9dd13c5d91c8ecd4c
Example 2:
# Run strace in the background:
nohup strace -s 256 -p `pidof nginx | sed -e 's/ /,/g'` 2>&1 -o /tmp/nginx-req.trace </dev/null >/dev/null 2>/dev/null &
# Watch output file:
watch -n 0.1 "awk '../../Host:/ {print \"pid: \" \$1 \", \" \"host: \" \$6}' /tmp/nginx-req.trace | sed 's/\\\r\\\n.*//'"
# Output example:
Every 0.1s: awk '../../Host:/ {print "pid: " $1 ", " "host: " $6}' /tmp/nginx-req.trace | sed 's/\\r\\n.*//'
pid: 31863, host: example.com
ngrep -d eth0 -qt 'HTTP' 'tcp'
tcpdump -ei eth0 -nn -A -s1500 -l | grep "User-Agent:"
# 1)
tcpdump -ei eth0 -s 0 -A -vv \
'tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x47455420' or 'tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504f5354'
# 2)
tcpdump -ei eth0 -s 0 -v -n -l | egrep -i "POST /|GET /|Host:"
ngrep -d eth0 "<server_name>" src host 10.10.252.1 and dst port 80
alias ng.test='nginx -t -c /etc/nginx/nginx.conf'
alias ng.stop='ng.test && systemctl stop nginx'
alias ng.reload='ng.test && systemctl reload nginx'
alias ng.reload='ng.test && kill -HUP $(cat /var/run/nginx.pid)'
# ... kill -HUP $(ps auxw | grep [n]ginx | grep master | awk '{print $2}')
alias ng.restart='ng.test && systemctl restart nginx'
alias ng.restart='ng.test && kill -QUIT $(cat /var/run/nginx.pid) && /usr/sbin/nginx'
# ... kill -QUIT $(ps auxw | grep [n]ginx | grep master | awk '{print $2}') ...
# 1) Generate file with htpasswd command:
htpasswd -c htpasswd_example.com.conf <username>
# 2) Include this file in specific context: (e.g. server):
server_name example.com;
...
# These directives are optional, only if we need them:
satisfy all;
deny 10.255.10.0/24;
allow 192.168.0.0/16;
allow 127.0.0.1;
deny all;
# It's important:
auth_basic "Restricted Area";
auth_basic_user_file /etc/nginx/acls/htpasswd_example.com.conf;
location / {
...
location /public/ {
auth_basic off;
}
...
Example 1:
# 1) File: /etc/nginx/acls/allow.map.conf
# Map module:
map $remote_addr $globals_internal_map_acl {
# Status code:
# - 0 = false
# - 1 = true
default 0;
### INTERNAL ###
10.255.10.0/24 1;
10.255.20.0/24 1;
10.255.30.0/24 1;
192.168.0.0/16 1;
}
# 2) Include this file in http context:
include /etc/nginx/acls/allow.map.conf;
# 3) Turn on in a specific context (e.g. location):
server_name example.com;
...
location / {
proxy_pass http://localhost:80;
client_max_body_size 10m;
}
location ~ ^/(backend|api|admin) {
if ($globals_internal_map_acl) {
set $pass 1;
}
if ($pass = 1) {
proxy_pass http://localhost:80;
client_max_body_size 10m;
}
if ($pass != 1) {
rewrite ^(.*) https://example.com;
}
...
Example 2:
# 1) File: /etc/nginx/acls/allow.geo.conf
# Geo module:
geo $globals_internal_geo_acl {
# Status code:
# - 0 = false
# - 1 = true
default 0;
### INTERNAL ###
10.255.10.0/24 1;
10.255.20.0/24 1;
10.255.30.0/24 1;
192.168.0.0/16 1;
}
# 2) Include this file in http context:
include /etc/nginx/acls/allow.geo.conf;
# 3) Turn on in a specific context (e.g. location):
server_name example.com;
...
location / {
proxy_pass http://localhost:80;
client_max_body_size 10m;
}
location ~ ^/(backend|api|admin) {
if ($globals_internal_geo_acl = 0) {
return 403;
}
proxy_pass http://localhost:80;
client_max_body_size 10m;
...
Example 3:
# 1) File: /etc/nginx/acls/allow.conf
### INTERNAL ###
allow 10.255.10.0/24;
allow 10.255.20.0/24;
allow 10.255.30.0/24;
allow 192.168.0.0/16;
### EXTERNAL ###
allow 35.228.233.xxx;
# 2) Include this file in http context:
include /etc/nginx/acls/allow.conf;
# 3) Turn on in a specific context (e.g. server):
server_name example.com;
include /etc/nginx/acls/allow.conf;
allow 35.228.233.xxx;
deny all;
...
Example 1:
# 1) File: /etc/nginx/limits.conf
map $http_referer $invalid_referer {
hostnames;
default 0;
# Invalid referrers:
"invalid.com" 1;
"~*spamdomain4.com" 1;
"~*.invalid\.org" 1;
}
# 2) Include this file in http context:
include /etc/nginx/limits.conf;
# 3) Turn on in a specific context (e.g. server):
server_name example.com;
if ($invalid_referer) { return 403; }
...
Example 2:
# 1) Turn on in a specific context (e.g. location):
location /check_status {
if ($http_referer ~ "spam1\.com|spam2\.com|spam3\.com") {
return 444;
}
...
How to test?
siege -b -r 2 -c 40 -v https:/example.com/storage/img/header.jpg -H "Referer: https://spamdomain4.com/"
** SIEGE 4.0.4
** Preparing 5 concurrent users for battle.
The server is now under siege...
HTTP/1.1 403 0.11 secs: 124 bytes ==> GET /storage/img/header.jpg
HTTP/1.1 403 0.12 secs: 124 bytes ==> GET /storage/img/header.jpg
HTTP/1.1 403 0.18 secs: 124 bytes ==> GET /storage/img/header.jpg
HTTP/1.1 403 0.18 secs: 124 bytes ==> GET /storage/img/header.jpg
HTTP/1.1 403 0.19 secs: 124 bytes ==> GET /storage/img/header.jpg
HTTP/1.1 403 0.10 secs: 124 bytes ==> GET /storage/img/header.jpg
HTTP/1.1 403 0.11 secs: 124 bytes ==> GET /storage/img/header.jpg
HTTP/1.1 403 0.11 secs: 124 bytes ==> GET /storage/img/header.jpg
HTTP/1.1 403 0.12 secs: 124 bytes ==> GET /storage/img/header.jpg
HTTP/1.1 403 0.12 secs: 124 bytes ==> GET /storage/img/header.jpg
...
Example 1:
# 1) File: /etc/nginx/limits.conf
map $http_referer $limit_ip_key_by_referer {
hostnames;
# It's important because if you set numeric value, e.g. 0 rate limiting rule will be catch all referers:
default "";
# Invalid referrers (we restrict them):
"invalid.com" $binary_remote_addr;
"~referer-xyz.com" $binary_remote_addr;
"~*spamdomain4.com" $binary_remote_addr;
"~*.invalid\.org" $binary_remote_addr;
}
limit_req_zone $limit_ip_key_by_referer zone=req_for_remote_addr_by_referer:1m rate=5r/s;
# 2) Include this file in http context:
include /etc/nginx/limits.conf;
# 3) Turn on in a specific context (e.g. server):
server_name example.com;
limit_req zone=req_for_remote_addr_by_referer burst=2;
...
How to test?
siege -b -r 2 -c 40 -v https:/example.com/storage/img/header.jpg -H "Referer: https://spamdomain4.com/"
** SIEGE 4.0.4
** Preparing 5 concurrent users for battle.
The server is now under siege...
HTTP/1.1 200 0.13 secs: 3174 bytes ==> GET /storage/img/header.jpg
HTTP/1.1 503 0.14 secs: 206 bytes ==> GET /storage/img/header.jpg
HTTP/1.1 503 0.15 secs: 206 bytes ==> GET /storage/img/header.jpg
HTTP/1.1 503 0.10 secs: 206 bytes ==> GET /storage/img/header.jpg
HTTP/1.1 503 0.10 secs: 206 bytes ==> GET /storage/img/header.jpg
HTTP/1.1 503 0.10 secs: 206 bytes ==> GET /storage/img/header.jpg
HTTP/1.1 200 0.63 secs: 3174 bytes ==> GET /storage/img/header.jpg
HTTP/1.1 200 1.13 secs: 3174 bytes ==> GET /storage/img/header.jpg
HTTP/1.1 200 1.00 secs: 3174 bytes ==> GET /storage/img/header.jpg
HTTP/1.1 200 1.04 secs: 3174 bytes ==> GET /storage/img/header.jpg
...
limit_req_zone $binary_remote_addr zone=req_for_remote_addr:64k rate=10r/m;
limit_req_zone
$binary_remote_addr
req_for_remote_addr
64k
(1024 IP addresses)0,16
request each second or 10
requests per minute (1
request every 6
second)Example of use:
location ~ /stats {
limit_req zone=req_for_remote_addr burst=5;
rate
* burst
in burst
seconds
5
requests:
0,16r/s
* 5
= 0.80
requests per 5
seconds10r/m
* 5
= 50
requests per 5
minutesTesting queue:
# siege -b -r 1 -c 12 -v https://x409.info/stats/
** SIEGE 4.0.4
** Preparing 12 concurrent users for battle.
The server is now under siege...
HTTP/1.1 200 * 0.20 secs: 2 bytes ==> GET /stats/
HTTP/1.1 503 0.20 secs: 1501 bytes ==> GET /stats/
HTTP/1.1 503 0.20 secs: 1501 bytes ==> GET /stats/
HTTP/1.1 503 0.21 secs: 1501 bytes ==> GET /stats/
HTTP/1.1 503 0.22 secs: 1501 bytes ==> GET /stats/
HTTP/1.1 503 0.22 secs: 1501 bytes ==> GET /stats/
HTTP/1.1 503 0.23 secs: 1501 bytes ==> GET /stats/
HTTP/1.1 200 * 6.22 secs: 2 bytes ==> GET /stats/
HTTP/1.1 200 * 12.24 secs: 2 bytes ==> GET /stats/
HTTP/1.1 200 * 18.27 secs: 2 bytes ==> GET /stats/
HTTP/1.1 200 * 24.30 secs: 2 bytes ==> GET /stats/
HTTP/1.1 200 * 30.32 secs: 2 bytes ==> GET /stats/
|
- burst=5
- 0,16r/s, 10r/m - 1r every 6 seconds
Transactions: 6 hits
Availability: 50.00 %
Elapsed time: 30.32 secs
Data transferred: 0.01 MB
Response time: 15.47 secs
Transaction rate: 0.20 trans/sec
Throughput: 0.00 MB/sec
Concurrency: 3.06
Successful transactions: 6
Failed transactions: 6
Longest transaction: 30.32
Shortest transaction: 0.20
limit_req_zone $binary_remote_addr zone=req_for_remote_addr:50m rate=2r/s;
limit_req_zone
$binary_remote_addr
req_for_remote_addr
50m
(800,000 IP addresses)2
request each second or 120
requests per minute (2
requests every 1
second)Example of use:
location ~ /stats {
limit_req zone=req_for_remote_addr burst=5 nodelay;
rate
* burst
in burst
seconds
5
requests
2r/s
* 5
= 10
requests per 5
seconds120r/m
* 5
= 600
requests per 5
minutesburst
parameter with nodelay
Testing queue:
# siege -b -r 1 -c 12 -v https://x409.info/stats/
** SIEGE 4.0.4
** Preparing 12 concurrent users for battle.
The server is now under siege...
HTTP/1.1 200 * 0.18 secs: 2 bytes ==> GET /stats/
HTTP/1.1 200 * 0.18 secs: 2 bytes ==> GET /stats/
HTTP/1.1 200 * 0.19 secs: 2 bytes ==> GET /stats/
HTTP/1.1 200 * 0.19 secs: 2 bytes ==> GET /stats/
HTTP/1.1 200 * 0.19 secs: 2 bytes ==> GET /stats/
HTTP/1.1 200 * 0.19 secs: 2 bytes ==> GET /stats/
HTTP/1.1 503 0.19 secs: 1501 bytes ==> GET /stats/
HTTP/1.1 503 0.19 secs: 1501 bytes ==> GET /stats/
HTTP/1.1 503 0.20 secs: 1501 bytes ==> GET /stats/
HTTP/1.1 503 0.21 secs: 1501 bytes ==> GET /stats/
HTTP/1.1 503 0.21 secs: 1501 bytes ==> GET /stats/
HTTP/1.1 503 0.22 secs: 1501 bytes ==> GET /stats/
|
- burst=5 with nodelay
- 2r/s, 120r/m - 1r every 0.5 second
Transactions: 6 hits
Availability: 50.00 %
Elapsed time: 0.23 secs
Data transferred: 0.01 MB
Response time: 0.39 secs
Transaction rate: 26.09 trans/sec
Throughput: 0.04 MB/sec
Concurrency: 10.17
Successful transactions: 6
Failed transactions: 6
Longest transaction: 0.22
Shortest transaction: 0.18
limit_conn_zone $binary_remote_addr zone=conn_for_remote_addr:1m;
limit_conn_zone
$binary_remote_addr
conn_for_remote_addr
1m
(16,000 IP addresses)Example of use:
location ~ /stats {
limit_conn conn_for_remote_addr 1;
1
connection from IP at the same timeTesting queue:
# siege -b -r 1 -c 100 -t 10s --no-parser https://x409.info/stats/
defaulting to time-based testing: 10 seconds
** SIEGE 4.0.4
** Preparing 100 concurrent users for battle.
The server is now under siege...
Lifting the server siege...
Transactions: 364 hits
Availability: 32.13 %
Elapsed time: 9.00 secs
Data transferred: 1.10 MB
Response time: 2.37 secs
Transaction rate: 40.44 trans/sec
Throughput: 0.12 MB/sec
Concurrency: 95.67
Successful transactions: 364
Failed transactions: 769
Longest transaction: 1.10
Shortest transaction: 0.38
www
prefixwww
to non-www
:server {
...
server_name www.domain.com;
# $scheme will get the http or https protocol:
return 301 $scheme://domain.com$request_uri;
}
non-www
to www
:server {
...
server_name domain.com;
# $scheme will get the http or https protocol:
return 301 $scheme://www.domain.com$request_uri;
}
POST data is passed in the body of the request, which gets dropped if you do a standard redirect.
Look at this:
DESCRITPION | PERMANENT | TEMPORARY |
---|---|---|
allows changing the request method from POST to GET | 301 | 302 |
does not allow changing the request method from POST to GET | 308 | 307 |
You can try with the HTTP status code 307, a RFC compliant browser should repeat the post request. You just need to write a NGINX rewrite rule with HTTP status code 307 or 308:
location /api {
# HTTP 307 only for POST requests:
if ($request_method = POST) {
return 307 https://api.example.com?request_uri;
}
# You can keep this for non-POST requests:
rewrite ^ https://api.example.com?request_uri permanent;
client_max_body_size 10m;
...
}
Example 1:
location ~* \.(?:ttf|ttc|otf|eot|woff|woff2)$ {
if ( $http_origin ~* (https?://(.+\.)?(domain1|domain2|domain3)\.(?:me|co|com)$) ) {
add_header "Access-Control-Allow-Origin" "$http_origin";
}
}
Example 2 (more slightly configuration; for GETs and POSTs):
location / {
if ($http_origin ~* (^https?://([^/]+\.)*(domainone|domaintwo)\.com$)) {
set $cors "true";
}
# Determine the HTTP request method used:
if ($request_method = 'GET') {
set $cors "${cors}get";
}
if ($request_method = 'POST') {
set $cors "${cors}post";
}
if ($cors = "true") {
# Catch all in case there's a request method we're not dealing with properly:
add_header 'Access-Control-Allow-Origin' "$http_origin";
}
if ($cors = "trueget") {
add_header 'Access-Control-Allow-Origin' "$http_origin";
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
}
if ($cors = "truepost") {
add_header 'Access-Control-Allow-Origin' "$http_origin";
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
}
}
Python 3.x:
python3 -m http.server 8000 --bind 127.0.0.1
Python 2.x:
python -m SimpleHTTPServer 8000
Python 3.x:
from http.server import HTTPServer, BaseHTTPRequestHandler
import ssl
httpd = HTTPServer(('localhost', 4443), BaseHTTPRequestHandler)
httpd.socket = ssl.wrap_socket (httpd.socket,
keyfile="path/to/key.pem",
certfile='path/to/cert.pem', server_side=True)
httpd.serve_forever()
Python 2.x:
import BaseHTTPServer, SimpleHTTPServer
import ssl
httpd = BaseHTTPServer.HTTPServer(('localhost', 4443),
SimpleHTTPServer.SimpleHTTPRequestHandler)
httpd.socket = ssl.wrap_socket (httpd.socket,
keyfile="path/tp/key.pem",
certfile='path/to/cert.pem', server_side=True)
httpd.serve_forever()
# _len: 2048, 4096
( _fd="private.key" ; _len="4096" ; \
openssl genrsa -out ${_fd} ${_len} )
( _fd="private.key" ; _fd_csr="request.csr" ; \
openssl req -out ${_fd_csr} -new -key ${_fd} )
( _fd="private.key" ; _fd_csr="request.csr" ; _fd_crt="cert.crt" ; \
openssl x509 -x509toreq -in ${_fd_crt} -out ${_fd_csr} -signkey ${_fd} )
( _fd="private.key" ; _fd_csr="request.csr" ; \
openssl req -new -sha256 -key ${_fd} -out ${_fd_csr} \
-config <(
cat <<-EOF
[req]
default_bits = 2048
prompt = no
default_md = sha256
req_extensions = req_ext
distinguished_name = dn
[ dn ]
C=<two-letter ISO abbreviation for your country>
ST=<state or province where your organization is legally located>
L=<city where your organization is legally located>
O=<legal name of your organization>
OU=<section of the organization>
CN=<fully qualified domain name>
[ req_ext ]
subjectAltName = @alt_names
[ alt_names ]
DNS.1 = <fully qualified domain name>
DNS.2 = <next domain>
DNS.3 = <next domain>
EOF
))
( _fd="private.key" ; _fd_csr="request.csr" ; _len="4096" ; \
openssl req -out ${_fd_csr} -new -newkey rsa:${_len} -nodes -keyout ${_fd} )
# _curve: prime256v1, secp521r1, secp384r1
( _fd="private.key" ; _curve="prime256v1" ; \
openssl ecparam -out ${_fd} -name ${_curve} -genkey )
# _curve: X25519
( _fd="private.key" ; _curve="x25519" ; \
openssl genpkey -algorithm ${_curve} -out ${_fd} )
# _curve: prime256v1, secp521r1, secp384r1
( _fd="domain.com.key" ; _fd_csr="domain.com.csr" ; _curve="prime256v1" ; \
openssl ecparam -out ${_fd} -name ${_curve} -genkey ; \
openssl req -new -key ${_fd} -out ${_fd_csr} -sha256 )
# _len: 2048, 4096
( _fd="domain.key" ; _fd_out="domain.crt" ; _len="4096" ; _days="365" ; \
openssl req -newkey rsa:${_len} -nodes \
-keyout ${_fd} -x509 -days ${_days} -out ${_fd_out} )
# _len: 2048, 4096
( _fd="domain.key" ; _fd_out="domain.crt" ; _days="365" ; \
openssl req -key ${_fd} -nodes \
-x509 -days ${_days} -out ${_fd_out} )
# _len: 2048, 4096
( _fd="domain.key" ; _fd_csr="domain.csr" ; _fd_out="domain.crt" ; _days="365" ; \
openssl x509 -signkey ${_fd} -nodes \
-in ${_fd_csr} -req -days ${_days} -out ${_fd_out} )
certbot certonly -d example.com -d www.example.com
certbot certonly --manual --preferred-challenges=dns -d example.com -d *.example.com
certbot certonly -d example.com -d www.example.com --rsa-key-size 4096
openssl dhparam -out /etc/nginx/ssl/dhparam_4096.pem 4096
( _fd_der="cert.crt" ; _fd_pem="cert.pem" ; \
openssl x509 -in ${_fd_der} -inform der -outform pem -out ${_fd_pem} )
( _fd_der="cert.crt" ; _fd_pem="cert.pem" ; \
openssl x509 -in ${_fd_pem} -outform der -out ${_fd_der} )
( _fd="private.key" ; \
openssl rsa -noout -text -in ${_fd} )
# 1)
( _fd="public.key" ; \
openssl pkey -noout -text -pubin -in ${_fd} )
# 2)
( _fd="private.key" ; \
openssl rsa -inform PEM -noout -in ${_fd} &> /dev/null ; \
if [ $? = 0 ] ; then echo -en "OK\n" ; fi )
( _fd="certificate.crt" ; # format: pem, cer, crt \
openssl x509 -noout -text -in ${_fd} )
( _fd_csr="request.csr" ; \
openssl req -text -noout -in ${_fd_csr} )
(openssl rsa -noout -modulus -in private.key | openssl md5 ; \
openssl x509 -noout -modulus -in certificate.crt | openssl md5) | uniq
# Install epel repository:
yum install epel-release
# or alternative:
# wget -c https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
# yum install epel-release-latest-7.noarch.rpm
# Install NGINX:
yum install nginx
# Install and enable scl:
yum install centos-release-scl
yum-config-manager --enable rhel-server-rhscl-7-rpms
# Install NGINX (rh-nginx14, rh-nginx16, rh-nginx18):
yum install rh-nginx16
# Enable NGINX from SCL:
scl enable rh-nginx16 bash
# Where:
# - <os_type> is: rhel or centos
cat > /etc/yum.repos.d/nginx.repo << __EOF__
[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/<os_type>/$releasever/$basearch/
gpgcheck=0
enabled=1
__EOF__
# Install NGINX:
yum install nginx
Check available flavors of NGINX before install. For more information please see this great answer by Thomas Ward.
# Install NGINX:
apt-get install nginx
# Where:
# - <os_type> is: debian or ubuntu
# - <os_release> is: xenial, bionic, jessie, stretch or other
cat > /etc/apt/sources.list.d/nginx.list << __EOF__
deb http://nginx.org/packages/<os_type>/ <os_release> nginx
deb-src http://nginx.org/packages/<os_type>/ <os_release> nginx
__EOF__
# Update packages list:
apt-get update
# Download the public key (or <pub_key> from your GPG error):
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys <pub_key>
# Install NGINX:
apt-get update
apt-get install nginx
The build is configured using the configure
command. The configure shell script attempts to guess correct values for various system-dependent variables used during compilation. It uses those values to create a Makefile
. Of course you can adjust certain environment variables to make configure able to find the packages like a zlib
or openssl
, and of many other options (paths, modules).
Before the beginning installation process please read these important articles which describes exactly the entire installation process and the parameters using the configure
command:
In this chapter I'll present three (very similar) methods of installation. They relate to:
Each of them is suited towards a high performance as well as high-concurrency applications. They work great as a high-end proxy servers too.
Look also on this short note about the system locations. That can be useful too:
/
/bin
- user programs/sbin
- system programs/lib
- shared libraries/usr
/usr/bin
- user programs/usr/sbin
- system programs/usr/lib
- shared libraries/usr/share
- manual pages, data/usr/local
/usr/local/bin
- user programs/usr/local/sbin
- system programs/usr/local/lib
- shared libraries/usr/local/share
- manual pages, dataInstallation from source consists of multiple steps. If you don't want to pass through all of them manually, you can run automated script. I created it to facilitate the whole installation process.
It supports Debian and RHEL like distributions.
This tool is located in lib/ngx_installer.sh
. Configuration file is in lib/ngx_installer.conf
. By default, it show prompt to confirm steps but you can disable it if you want:
cd lib/
export NGX_PROMPT=0 ; bash ngx_installer.sh
There are currently two versions of NGINX:
You can download NGINX source code from an official read-only mirrors:
Detailed instructions about download and compile the NGINX sources can be found later in the handbook.
Mandatory requirements:
Download, compile and install or install prebuilt packages from repository of your distribution.
OpenResty's LuaJIT uses its own branch of LuaJIT with various important bug fixes and optimizations for OpenResty's use cases.
I also use Cloudflare Zlib version due to performance. See below articles:
If you download and compile above sources the good point is to install additional packages (dependent on the system version) before building NGINX:
Debian Like | RedHat Like | Comment |
---|---|---|
gcc make build-essential linux-headers* bison |
gcc gcc-c++ kernel-devel bison |
|
perl libperl-dev libphp-embed |
perl perl-devel perl-ExtUtils-Embed |
|
libssl-dev * |
openssl-devel * |
|
zlib1g-dev * |
zlib-devel * |
|
libpcre2-dev * |
pcre-devel * |
|
libluajit-5.1-dev * |
luajit-devel * |
|
libxslt-dev |
libxslt libxslt-devel |
|
libgd-dev |
gd gd-devel |
|
libgeoip-dev |
GeoIP-devel |
|
libxml2-dev |
libxml2-devel |
|
libexpat-dev |
expat-devel |
|
libgoogle-perftools-dev libgoogle-perftools4 |
gperftools-devel |
|
cpio |
||
gettext-devel |
||
autoconf |
autoconf |
for jemalloc from sources |
libjemalloc1 libjemalloc-dev * |
jemalloc jemalloc-devel * |
for jemalloc |
libpam0g-dev |
pam-devel |
for ngx_http_auth_pam_module |
jq |
jq |
for http error pages generator |
* If you don't use from sources.
Shell one-liners example:
# Ubuntu/Debian
apt-get install gcc make build-essential bison perl libperl-dev libphp-embed libssl-dev zlib1g-dev libpcre2-dev libluajit-5.1-dev libxslt-dev libgd-dev libgeoip-dev libxml2-dev libexpat-dev libgoogle-perftools-dev libgoogle-perftools4 autoconf jq
# RedHat/CentOS
yum install gcc gcc-c++ kernel-devel bison perl perl-devel perl-ExtUtils-Embed openssl-devel zlib-devel pcre-devel luajit-devel libxslt libxslt-devel gd gd-devel GeoIP-devel libxml2-devel expat-devel gperftools-devel cpio gettext-devel autoconf jq
Not all external modules can work properly with your currently NGINX version. You should read the documentation of each module before adding it to the modules list. You should also to check what version of module is compatible with your NGINX release.
Modules can be compiled as a shared object (*.so
file) and then dynamically loaded into NGINX at runtime (--add-dynamic-module
). On the other hand you can also built them into NGINX at compile time and linked to the NGINX binary statically (--add-module
).
I mixed both variants because some of the modules are built-in automatically even if I try them to be compiled as a dynamic modules (they are not support dynamic linking).
You can download external modules from:
A short description of the modules that I used (not only) in this step-by-step tutorial:
ngx_devel_kit
** - adds additional generic tools that module developers can use in their own moduleslua-nginx-module
- embed the Power of Lua into NGINXset-misc-nginx-module
- various set_xxx
directives added to NGINX's rewrite moduleecho-nginx-module
- module for bringing the power of echo
, sleep
, time
and more to NGINX's config fileheaders-more-nginx-module
- set, add, and clear arbitrary output headersreplace-filter-nginx-module
- streaming regular expression replacement in response bodiesarray-var-nginx-module
- add supports for array-typed variables to NGINX config filesencrypted-session-nginx-module
- encrypt and decrypt NGINX variable valuesnginx-module-sysguard
- module to protect servers when system load or memory use goes too highnginx-access-plus
- allows limiting access to certain http request methods and client addressesngx_http_substitutions_filter_module
- can do both regular expression and fixed string substitutionsnginx-sticky-module-ng
- module to add a sticky cookie to be always forwarded to the samenginx-module-vts
- Nginx virtual host traffic status modulengx_brotli
- module for Brotli compressionngx_http_naxsi_module
- is an open-source, high performance, low rules maintenance WAF for NGINXngx_http_delay_module
- allows to delay requests for a given timenginx-backtrace
* - module to dump backtrace when a worker process exits abnormallyngx_debug_pool
* - provides access to information of memory usage for NGINX memory poolngx_debug_timer
* - provides access to information of timer usage for NGINXnginx_upstream_check_module
* - health checks upstreams for NGINXnginx-http-footer-filter
* - module that prints some text in the footer of a request upstream servermemc-nginx-module
- extended version of the standard Memcached modulenginx-rtmp-module
- NGINX-based Media Streaming Serverngx-fancyindex
- generates of file listings, like the built-in autoindex module does, but adding a touch of stylengx_log_if
- allows you to control when not to write down access lognginx-http-user-agent
- module to match browsers and crawlersngx_http_auth_pam_module
- module to use PAM for simple http authenticationngx_http_google_filter_module
- is a filter module which makes google mirror much easier to deploynginx-push-stream-module
- a pure stream http push technology for your Nginx setupnginx_tcp_proxy_module
- add the feature of tcp proxy with nginx, with health check and status monitor* Available in Tengine Web Server (but these modules may have been updated/patched by Tengine Team).
** Is already being used in quite a few third party modules.
Someting about compiler and linker options. Out of the box you probably do not need to provide any flags yourself, the configure script should detect automatically some reasonable defaults. However, in order to optimize for speed and/or security, you should probably provide a few compiler flags. See this recommendations by RedHat.
You should also read Compilation and Installation for OpenSSL.
There are examples:
# 1)
# --with-cc-opt="-I/usr/local/include -I${OPENSSL_INC} -I${LUAJIT_INC} -I${JEMALLOC_INC} -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC"
# 2)
# --with-cc-opt="-I/usr/local/include -m64 -march=native -DTCP_FASTOPEN=23 -g -O3 -fstack-protector-strong -flto -fuse-ld=gold --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wno-deprecated-declarations -gsplit-dwarf"
# Example of use linker options:
# 1)
# --with-ld-opt="-Wl,-E -L/usr/local/lib -ljemalloc -lpcre -Wl,-rpath,/usr/local/lib,-z,relro -Wl,-z,now -pie"
# 2)
# --with-ld-opt="-L/usr/local/lib -ljemalloc -Wl,-lpcre -Wl,-z,relro -Wl,-rpath,/usr/local/lib"
SystemTap is a scripting language and tool for dynamically instrumenting running production Linux kernel-based operating systems. It's required for
openresty-systemtap-toolkit
for OpenResty.
It's good all-in-one tutorial for install and configure SystemTap on CentOS 7/Ubuntu distributions.
cd /opt
git clone --depth 1 https://github.com/openresty/openresty-systemtap-toolkit
# RHEL/CentOS
yum install yum-utils
yum --enablerepo=base-debuginfo install kernel-devel-$(uname -r) kernel-headers-$(uname -r) kernel-debuginfo-$(uname -r) kernel-debuginfo-common-x86_64-$(uname -r)
yum --enablerepo=base-debuginfo install systemtap systemtap-debuginfo
reboot
# Run this commands for testing SystemTap:
stap -v -e 'probe vfs.read {printf("read performed\n"); exit()}'
stap -v -e 'probe begin { printf("Hello, World!\n"); exit() }'
For installation SystemTap on Ubuntu/Debian:
Set NGINX version (I use stable and newest release):
export ngx_version="1.16.0"
Set temporary variables:
ngx_src="../../usr/local/src"
ngx_base="${ngx_src}/nginx-${ngx_version}"
ngx_master="${ngx_base}/master"
ngx_modules="${ngx_base}/modules"
Create directories:
for i in "$ngx_base" "${ngx_master}" "$ngx_modules" ; do
mkdir "$i"
done
In my configuration I used all prebuilt dependencies without
libssl-dev
,zlib1g-dev
,libluajit-5.1-dev
andlibpcre2-dev
because I compiled them manually - for TLS 1.3 support and with OpenResty recommendation for LuaJIT.
Install prebuilt packages, export variables and set symbolic link:
# It's important and required, regardless of chosen sources:
yum install gcc gcc-c++ kernel-devel bison perl perl-devel perl-ExtUtils-Embed libxslt libxslt-devel gd gd-devel GeoIP-devel libxml2-devel expat-devel gperftools-devel cpio gettext-devel autoconf jq
# In this example we use sources for all below packages so we do not install them:
yum install openssl-devel zlib-devel pcre-devel luajit-devel
# For LuaJIT (libluajit-5.1-dev):
export LUAJIT_LIB="../../usr/local/x86_64-linux-gnu"
export LUAJIT_INC="../../usr/include/luajit-2.1"
ln -s /usr/lib/x86_64-linux-gnu/libluajit-5.1.so.2 /usr/local/lib/liblua.so
Remember to build
sregex
also if you use above steps.
Or download and compile them:
PCRE:
cd "${ngx_src}"
export pcre_version="8.42"
export PCRE_SRC="${ngx_src}/pcre-${pcre_version}"
export PCRE_LIB="../../usr/local/lib"
export PCRE_INC="../../usr/local/include"
wget https://ftp.pcre.org/pub/pcre/pcre-${pcre_version}.tar.gz && tar xzvf pcre-${pcre_version}.tar.gz
cd "$PCRE_SRC"
./configure
make -j2 && make test
make install
Zlib:
# I recommend to use Cloudflare Zlib version (cloudflare/zlib) instead an original Zlib (zlib.net), but both installation methods are similar:
cd "${ngx_src}"
export ZLIB_SRC="${ngx_src}/zlib"
export ZLIB_LIB="../../usr/local/lib"
export ZLIB_INC="../../usr/local/include"
# For original Zlib:
# export zlib_version="1.2.11"
# wget http://www.zlib.net/zlib-${zlib_version}.tar.gz && tar xzvf zlib-${zlib_version}.tar.gz
# cd "${ZLIB_SRC}-${zlib_version}"
# For Cloudflare Zlib:
git clone --depth 1 https://github.com/cloudflare/zlib
cd "$ZLIB_SRC"
./configure
make -j2 && make test
make install
OpenSSL:
cd "${ngx_src}"
export openssl_version="1.1.1b"
export OPENSSL_SRC="${ngx_src}/openssl-${openssl_version}"
export OPENSSL_DIR="../../usr/local/openssl-${openssl_version}"
export OPENSSL_LIB="${OPENSSL_DIR}/lib"
export OPENSSL_INC="${OPENSSL_DIR}/include"
wget https://www.openssl.org/source/openssl-${openssl_version}.tar.gz && tar xzvf openssl-${openssl_version}.tar.gz
cd "${ngx_src}/openssl-${openssl_version}"
# Please run this and add as a compiler param:
export __GCC_SSL=("__SIZEOF_INT128__:enable-ec_nistp_64_gcc_128")
for _cc_opt in "${__GCC_SSL[@]}" ; do
_cc_key=$(echo "$_cc_opt" | cut -d ":" -f1)
_cc_value=$(echo "$_cc_opt" | cut -d ":" -f2)
if [[ ! $(gcc -dM -E - </dev/null | grep -q "$_cc_key") ]] ; then
echo -en "$_cc_value is supported on this machine\n"
_openssl_gcc+="$_cc_value "
fi
done
./config --prefix="$OPENSSL_DIR" --openssldir="$OPENSSL_DIR" shared zlib no-ssl3 no-weak-ssl-ciphers -DOPENSSL_NO_HEARTBEATS -fstack-protector-strong "$_openssl_gcc"
make -j2 && make test
make install
# Setup PATH environment variables:
cat > /etc/profile.d/openssl.sh << __EOF__
#!/bin/sh
export PATH=${OPENSSL_DIR}/bin:${PATH}
export LD_LIBRARY_PATH=${OPENSSL_DIR}/lib:${LD_LIBRARY_PATH}
__EOF__
chmod +x /etc/profile.d/openssl.sh && source /etc/profile.d/openssl.sh
# To make the OpenSSL 1.1.1b version visible globally first:
mv /usr/bin/openssl /usr/bin/openssl-old
ln -s ${OPENSSL_DIR}/bin/openssl /usr/bin/openssl
cat > /etc/ld.so.conf.d/openssl.conf << __EOF__
${OPENSSL_DIR}/lib
__EOF__
LuaJIT:
# I recommend to use OpenResty's branch (openresty/luajit2) instead LuaJIT (LuaJIT/LuaJIT), but both installation methods are similar:
cd "${ngx_src}"
export LUAJIT_SRC="${ngx_src}/luajit2"
export LUAJIT_LIB="../../usr/local/lib"
export LUAJIT_INC="../../usr/local/include/luajit-2.1"
# For originall LuaJIT:
# git clone http://luajit.org/git/luajit-2.0 luajit2
# cd "$LUAJIT_SRC"
# For OpenResty's LuaJIT:
git clone --depth 1 https://github.com/openresty/luajit2
cd "$LUAJIT_SRC"
make && make install
ln -s /usr/local/lib/libluajit-5.1.so.2.1.0 /usr/local/lib/liblua.so
Required for
replace-filter-nginx-module
module.
cd "${ngx_src}"
git clone --depth 1 https://github.com/openresty/sregex
cd "${ngx_src}/sregex"
make && make install
jemalloc:
To verify
jemalloc
in use:lsof -n | grep jemalloc
.
cd "${ngx_src}"
export JEMALLOC_SRC="${ngx_src}/jemalloc"
export JEMALLOC_INC="../../usr/local/include/jemalloc"
git clone --depth 1 https://github.com/jemalloc/jemalloc
cd "$JEMALLOC_SRC"
./autogen.sh
make && make install
Update links and cache to the shared libraries for both types of installation:
ldconfig
cd "${ngx_base}"
wget https://nginx.org/download/nginx-${ngx_version}.tar.gz
# or alternative:
# git clone --depth 1 https://github.com/nginx/nginx master
tar zxvf nginx-${ngx_version}.tar.gz -C "${ngx_master}" --strip 1
cd "${ngx_modules}"
for i in \
https://github.com/simplresty/ngx_devel_kit \
https://github.com/openresty/lua-nginx-module \
https://github.com/openresty/set-misc-nginx-module \
https://github.com/openresty/echo-nginx-module \
https://github.com/openresty/headers-more-nginx-module \
https://github.com/openresty/replace-filter-nginx-module \
https://github.com/openresty/array-var-nginx-module \
https://github.com/openresty/encrypted-session-nginx-module \
https://github.com/vozlt/nginx-module-sysguard \
https://github.com/nginx-clojure/nginx-access-plus \
https://github.com/yaoweibin/ngx_http_substitutions_filter_module \
https://bitbucket.org/nginx-goodies/nginx-sticky-module-ng \
https://github.com/vozlt/nginx-module-vts \
https://github.com/google/ngx_brotli ; do
git clone --depth 1 "$i"
done
wget http://mdounin.ru/hg/ngx_http_delay_module/archive/tip.tar.gz -O delay-module.tar.gz
mkdir delay-module && tar xzvf delay-module.tar.gz -C delay-module --strip 1
For ngx_brotli
:
cd "${ngx_modules}/ngx_brotli"
git submodule update --init
I also use some modules from Tengine:
ngx_backtrace_module
ngx_debug_pool
ngx_debug_timer
ngx_http_upstream_check_module
ngx_http_footer_filter_module
cd "${ngx_modules}"
git clone --depth 1 https://github.com/alibaba/tengine
If you use NAXSI:
cd "${ngx_modules}"
git clone --depth 1 https://github.com/nbs-system/naxsi
cd "${ngx_master}"
# - you can also build NGINX without 3rd party modules
# - remember about compiler and linker options
# - don't set values for --with-openssl, --with-pcre, and --with-zlib if you select prebuilt packages for them
./configure --prefix=/etc/nginx \
--conf-path=/etc/nginx/nginx.conf \
--sbin-path=/usr/sbin/nginx \
--pid-path=/var/run/nginx.pid \
--lock-path=/var/run/nginx.lock \
--user=nginx \
--group=nginx \
--modules-path=/etc/nginx/modules \
--error-log-path=/var/log/nginx/error.log \
--http-log-path=/var/log/nginx/access.log \
--http-client-body-temp-path=/var/cache/nginx/client_temp \
--http-proxy-temp-path=/var/cache/nginx/proxy_temp \
--http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp \
--http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp \
--http-scgi-temp-path=/var/cache/nginx/scgi_temp \
--with-compat \
--with-debug \
--with-file-aio \
--with-threads \
--with-stream \
--with-stream_realip_module \
--with-stream_ssl_module \
--with-stream_ssl_preread_module \
--with-http_addition_module \
--with-http_auth_request_module \
--with-http_degradation_module \
--with-http_geoip_module \
--with-http_gunzip_module \
--with-http_gzip_static_module \
--with-http_image_filter_module \
--with-http_perl_module \
--with-http_random_index_module \
--with-http_realip_module \
--with-http_secure_link_module \
--with-http_ssl_module \
--with-http_stub_status_module \
--with-http_sub_module \
--with-http_v2_module \
--with-google_perftools_module \
--with-openssl=${OPENSSL_SRC} \
--with-openssl-opt="shared zlib no-ssl3 no-weak-ssl-ciphers -DOPENSSL_NO_HEARTBEATS -fstack-protector-strong ${_openssl_gcc}" \
--with-pcre=${PCRE_SRC} \
--with-pcre-jit \
--with-zlib=${ZLIB_SRC} \
--without-http-cache \
--without-http_memcached_module \
--without-mail_pop3_module \
--without-mail_imap_module \
--without-mail_smtp_module \
--without-http_fastcgi_module \
--without-http_scgi_module \
--without-http_uwsgi_module \
--add-module=${ngx_modules}/ngx_devel_kit \
--add-module=${ngx_modules}/encrypted-session-nginx-module \
--add-module=${ngx_modules}/nginx-access-plus/src/c \
--add-module=${ngx_modules}/ngx_http_substitutions_filter_module \
--add-module=${ngx_modules}/nginx-sticky-module-ng \
--add-module=${ngx_modules}/nginx-module-vts \
--add-module=${ngx_modules}/ngx_brotli \
--add-module=${ngx_modules}/tengine/modules/ngx_backtrace_module \
--add-module=${ngx_modules}/tengine/modules/ngx_debug_pool \
--add-module=${ngx_modules}/tengine/modules/ngx_debug_timer \
--add-module=${ngx_modules}/tengine/modules/ngx_http_footer_filter_module \
--add-module=${ngx_modules}/tengine/modules/ngx_http_upstream_check_module \
--add-module=${ngx_modules}/tengine/modules/ngx_slab_stat \
--add-dynamic-module=${ngx_modules}/lua-nginx-module \
--add-dynamic-module=${ngx_modules}/set-misc-nginx-module \
--add-dynamic-module=${ngx_modules}/echo-nginx-module \
--add-dynamic-module=${ngx_modules}/headers-more-nginx-module \
--add-dynamic-module=${ngx_modules}/replace-filter-nginx-module \
--add-dynamic-module=${ngx_modules}/array-var-nginx-module \
--add-dynamic-module=${ngx_modules}/nginx-module-sysguard \
--add-dynamic-module=${ngx_modules}/delay-module \
--add-dynamic-module=${ngx_modules}/naxsi/naxsi_src \
--with-cc-opt="-I/usr/local/include -m64 -march=native -DTCP_FASTOPEN=23 -g -O3 -fstack-protector-strong -flto -fuse-ld=gold --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wno-deprecated-declarations -gsplit-dwarf" \
--with-ld-opt="-L/usr/local/lib -ljemalloc -Wl,-lpcre -Wl,-z,relro -Wl,-rpath,/usr/local/lib"
make -j2 && make test
make install
ldconfig
Check NGINX version:
nginx -v
nginx version: nginx/1.16.0
And list all files in /etc/nginx
:
.
├── fastcgi.conf
├── fastcgi.conf.default
├── fastcgi_params
├── fastcgi_params.default
├── html
│ ├── 50x.html
│ └── index.html
├── koi-utf
├── koi-win
├── mime.types
├── mime.types.default
├── modules
│ ├── ngx_http_array_var_module.so
│ ├── ngx_http_delay_module.so
│ ├── ngx_http_echo_module.so
│ ├── ngx_http_headers_more_filter_module.so
│ ├── ngx_http_lua_module.so
│ ├── ngx_http_naxsi_module.so
│ ├── ngx_http_replace_filter_module.so
│ ├── ngx_http_set_misc_module.so
│ └── ngx_http_sysguard_module.so
├── nginx.conf
├── nginx.conf.default
├── scgi_params
├── scgi_params.default
├── uwsgi_params
├── uwsgi_params.default
└── win-utf
2 directories, 26 files
Create a system user/group:
# Ubuntu/Debian
adduser --system --home /non-existent --no-create-home --shell /usr/sbin/nologin --disabled-login --disabled-password --gecos "nginx user" --group nginx
# RedHat/CentOS
groupadd -r -g 920 nginx
useradd --system --home-dir /non-existent --no-create-home --shell /usr/sbin/nologin --uid 920 --gid nginx nginx
passwd -l nginx
Create required directories:
for i in \
/var/www \
/var/log/nginx \
/var/cache/nginx ; do
mkdir -p "$i" && chown -R nginx:nginx "$i"
done
Include the necessary error pages:
You can also define them e.g. in
/etc/nginx/errors.conf
or other file and attach it as needed in server contexts.
/etc/nginx/html
50x.html index.html
Update modules list and include modules.conf
to your configuration:
_mod_dir="../../etc/nginx/modules"
:>"${_mod_dir}.conf"
for _module in $(ls "${_mod_dir}/") ; do echo -en "load_module\t\t${_mod_dir}/$_module;\n" >> "${_mod_dir}.conf" ; done
Create logrotate
configuration:
cat > /etc/logrotate.d/nginx << __EOF__
/var/log/nginx/*.log {
daily
missingok
rotate 14
compress
delaycompress
notifempty
create 0640 nginx nginx
sharedscripts
prerotate
if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
run-parts /etc/logrotate.d/httpd-prerotate; \
fi \
endscript
postrotate
invoke-rc.d nginx reload >/dev/null 2>&1
endscript
}
__EOF__
Add systemd service:
cat > /lib/systemd/system/nginx.service << __EOF__
# Stop dance for nginx
# =======================
#
# ExecStop sends SIGSTOP (graceful stop) to the nginx process.
# If, after 5s (--retry QUIT/5) nginx is still running, systemd takes control
# and sends SIGTERM (fast shutdown) to the main process.
# After another 5s (TimeoutStopSec=5), and if nginx is alive, systemd sends
# SIGKILL to all the remaining processes in the process group (KillMode=mixed).
#
# nginx signals reference doc:
# http://nginx.org/en/docs/control.html
#
[Unit]
Description=A high performance web server and a reverse proxy server
Documentation=man:nginx(8)
After=network.target
[Service]
Type=forking
PIDFile=/run/nginx.pid
ExecStartPre=/usr/sbin/nginx -t -q -g 'daemon on; master_process on;'
ExecStart=/usr/sbin/nginx -g 'daemon on; master_process on;'
ExecReload=/usr/sbin/nginx -g 'daemon on; master_process on;' -s reload
ExecStop=-/sbin/start-stop-daemon --quiet --stop --retry QUIT/5 --pidfile /run/nginx.pid
TimeoutStopSec=5
KillMode=mixed
[Install]
WantedBy=multi-user.target
__EOF__
Reload systemd manager configuration:
systemctl daemon-reload
Enable NGINX service:
systemctl enable nginx
Test NGINX configuration:
nginx -t -c /etc/nginx/nginx.conf
OpenResty is a full-fledged web application server by bundling the standard nginx core, lots of 3rd-party nginx modules, as well as most of their external dependencies.
This bundle is maintained by Yichun Zhang (agentzh).
OpenResty is a more than web server. I would call it a superset of the NGINX web server. OpenResty comes with LuaJIT, a just-in-time compiler for the Lua scripting language and many Lua libraries, lots of high quality 3rd-party NGINX modules, and most of their external dependencies.
OpenResty has good quality and performance. For me, the ability to run Lua scripts from within is also really great.
Set the OpenResty version (I use newest and stable release):
export ngx_version="1.15.8.1"
Set temporary variables:
ngx_src="../../usr/local/src"
ngx_base="${ngx_src}/openresty-${ngx_version}"
ngx_master="${ngx_base}/master"
ngx_modules="${ngx_base}/modules"
Create directories:
for i in "$ngx_base" "${ngx_master}" "$ngx_modules" ; do
mkdir "$i"
done
In my configuration I used all prebuilt dependencies without
libssl-dev
,zlib1g-dev
, andlibpcre2-dev
because I compiled them manually - for TLS 1.3 support. In addition, LuaJIT comes with OpenResty.
Install prebuilt packages, export variables and set symbolic link:
# It's important and required, regardless of chosen sources:
yum install gcc gcc-c++ kernel-devel bison perl perl-devel perl-ExtUtils-Embed libxslt libxslt-devel gd gd-devel GeoIP-devel libxml2-devel expat-devel gperftools-devel cpio gettext-devel autoconf jq
# In this example we use sources for all below packages so we do not install them:
yum install openssl-devel zlib-devel pcre-devel
Remember to build
sregex
also if you use above steps.
Or download and compile them:
PCRE:
cd "${ngx_src}"
export pcre_version="8.42"
export PCRE_SRC="${ngx_base}/pcre-${pcre_version}"
export PCRE_LIB="../../usr/local/lib"
export PCRE_INC="../../usr/local/include"
wget https://ftp.pcre.org/pub/pcre/pcre-${pcre_version}.tar.gz && tar xzvf pcre-${pcre_version}.tar.gz
cd "$PCRE_SRC"
./configure
make -j2 && make test
make install
Zlib:
# I recommend to use Cloudflare Zlib version (cloudflare/zlib) instead an original Zlib (zlib.net), but both installation methods are similar:
cd "${ngx_src}"
export ZLIB_SRC="${ngx_src}/zlib"
export ZLIB_LIB="../../usr/local/lib"
export ZLIB_INC="../../usr/local/include"
# For original Zlib:
# export zlib_version="1.2.11"
# wget http://www.zlib.net/zlib-${zlib_version}.tar.gz && tar xzvf zlib-${zlib_version}.tar.gz
# cd "${ZLIB_SRC}-${zlib_version}"
# For Cloudflare Zlib:
git clone --depth 1 https://github.com/cloudflare/zlib
cd "$ZLIB_SRC"
./configure
make -j2 && make test
make install
OpenSSL:
cd "${ngx_src}"
export openssl_version="1.1.1b"
export OPENSSL_SRC="${ngx_src}/openssl-${openssl_version}"
export OPENSSL_DIR="../../usr/local/openssl-${openssl_version}"
export OPENSSL_LIB="${OPENSSL_DIR}/lib"
export OPENSSL_INC="${OPENSSL_DIR}/include"
wget https://www.openssl.org/source/openssl-${openssl_version}.tar.gz && tar xzvf openssl-${openssl_version}.tar.gz
cd "${ngx_src}/openssl-${openssl_version}"
# Please run this and add as a compiler param:
export __GCC_SSL=("__SIZEOF_INT128__:enable-ec_nistp_64_gcc_128")
for _cc_opt in "${__GCC_SSL[@]}" ; do
_cc_key=$(echo "$_cc_opt" | cut -d ":" -f1)
_cc_value=$(echo "$_cc_opt" | cut -d ":" -f2)
if [[ ! $(gcc -dM -E - </dev/null | grep -q "$_cc_key") ]] ; then
echo -en "$_cc_value is supported on this machine\n"
_openssl_gcc+="$_cc_value "
fi
done
./config --prefix="$OPENSSL_DIR" --openssldir="$OPENSSL_DIR" shared zlib no-ssl3 no-weak-ssl-ciphers -DOPENSSL_NO_HEARTBEATS -fstack-protector-strong "$_openssl_gcc"
make -j2 && make test
make install
# Setup PATH environment variables:
cat > /etc/profile.d/openssl.sh << __EOF__
#!/bin/sh
export PATH=${OPENSSL_DIR}/bin:${PATH}
export LD_LIBRARY_PATH=${OPENSSL_DIR}/lib:${LD_LIBRARY_PATH}
__EOF__
chmod +x /etc/profile.d/openssl.sh && source /etc/profile.d/openssl.sh
# To make the OpenSSL 1.1.1b version visible globally first:
mv /usr/bin/openssl /usr/bin/openssl-old
ln -s ${OPENSSL_DIR}/bin/openssl /usr/bin/openssl
cat > /etc/ld.so.conf.d/openssl.conf << __EOF__
${OPENSSL_DIR}/lib
__EOF__
Required for
replace-filter-nginx-module
module.
cd "${ngx_src}"
git clone --depth 1 https://github.com/openresty/sregex
cd "${ngx_src}/sregex"
make && make install
jemalloc:
To verify
jemalloc
in use:lsof -n | grep jemalloc
.
cd "${ngx_src}"
export JEMALLOC_SRC="../../usr/local/src/jemalloc"
export JEMALLOC_INC="../../usr/local/include/jemalloc"
git clone --depth 1 https://github.com/jemalloc/jemalloc
cd "$JEMALLOC_SRC"
./autogen.sh
make && make install
Update links and cache to the shared libraries for both types of installation:
ldconfig
cd "${ngx_base}"
wget https://openresty.org/download/openresty-${ngx_version}.tar.gz
tar zxvf openresty-${ngx_version}.tar.gz -C "${ngx_master}" --strip 1
cd "${ngx_modules}"
for i in \
https://github.com/openresty/replace-filter-nginx-module \
https://github.com/vozlt/nginx-module-sysguard \
https://github.com/nginx-clojure/nginx-access-plus \
https://github.com/yaoweibin/ngx_http_substitutions_filter_module \
https://bitbucket.org/nginx-goodies/nginx-sticky-module-ng \
https://github.com/vozlt/nginx-module-vts \
https://github.com/google/ngx_brotli ; do
git clone --depth 1 "$i"
done
wget http://mdounin.ru/hg/ngx_http_delay_module/archive/tip.tar.gz -O delay-module.tar.gz
mkdir delay-module && tar xzvf delay-module.tar.gz -C delay-module --strip 1
For ngx_brotli
:
cd "${ngx_modules}/ngx_brotli"
git submodule update --init
I also use some modules from Tengine:
ngx_backtrace_module
ngx_debug_pool
ngx_debug_timer
ngx_http_upstream_check_module
ngx_http_footer_filter_module
cd "${ngx_modules}"
git clone --depth 1 https://github.com/alibaba/tengine
If you use NAXSI:
cd "${ngx_modules}"
git clone --depth 1 https://github.com/nbs-system/naxsi
cd "${ngx_master}"
# - you can also build OpenResty without 3rd party modules
# - remember about compiler and linker options
# - don't set values for --with-openssl, --with-pcre, and --with-zlib if you select prebuilt packages for them
./configure --prefix=/etc/nginx \
--conf-path=/etc/nginx/nginx.conf \
--sbin-path=/usr/sbin/nginx \
--pid-path=/var/run/nginx.pid \
--lock-path=/var/run/nginx.lock \
--user=nginx \
--group=nginx \
--modules-path=/etc/nginx/modules \
--error-log-path=/var/log/nginx/error.log \
--http-log-path=/var/log/nginx/access.log \
--http-client-body-temp-path=/var/cache/nginx/client_temp \
--http-proxy-temp-path=/var/cache/nginx/proxy_temp \
--http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp \
--http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp \
--http-scgi-temp-path=/var/cache/nginx/scgi_temp \
--with-compat \
--with-debug \
--with-file-aio \
--with-threads \
--with-stream \
--with-stream_geoip_module \
--with-stream_realip_module \
--with-stream_ssl_module \
--with-stream_ssl_preread_module \
--with-http_addition_module \
--with-http_auth_request_module \
--with-http_degradation_module \
--with-http_geoip_module \
--with-http_gunzip_module \
--with-http_gzip_static_module \
--with-http_image_filter_module \
--with-http_perl_module \
--with-http_random_index_module \
--with-http_realip_module \
--with-http_secure_link_module \
--with-http_slice_module \
--with-http_ssl_module \
--with-http_stub_status_module \
--with-http_sub_module \
--with-http_v2_module \
--with-google_perftools_module \
--with-luajit \
--with-openssl=${OPENSSL_SRC} \
--with-openssl-opt="shared zlib no-ssl3 no-weak-ssl-ciphers -DOPENSSL_NO_HEARTBEATS -fstack-protector-strong ${_openssl_gcc}" \
--with-pcre=${PCRE_SRC} \
--with-pcre-jit \
--with-zlib=${ZLIB_SRC} \
--without-http-cache \
--without-http_memcached_module \
--without-http_redis2_module \
--without-http_redis_module \
--without-http_rds_json_module \
--without-http_rds_csv_module \
--without-lua_redis_parser \
--without-lua_rds_parser \
--without-lua_resty_redis \
--without-lua_resty_memcached \
--without-lua_resty_mysql \
--without-lua_resty_websocket \
--without-mail_pop3_module \
--without-mail_imap_module \
--without-mail_smtp_module \
--without-http_fastcgi_module \
--without-http_scgi_module \
--without-http_uwsgi_module \
--add-module=${ngx_modules}/nginx-access-plus/src/c \
--add-module=${ngx_modules}/ngx_http_substitutions_filter_module \
--add-module=${ngx_modules}/nginx-module-vts \
--add-module=${ngx_modules}/ngx_brotli \
--add-module=${ngx_modules}/tengine/modules/ngx_backtrace_module \
--add-module=${ngx_modules}/tengine/modules/ngx_debug_pool \
--add-module=${ngx_modules}/tengine/modules/ngx_debug_timer \
--add-module=${ngx_modules}/tengine/modules/ngx_http_footer_filter_module \
--add-module=${ngx_modules}/tengine/modules/ngx_http_upstream_check_module \
--add-module=${ngx_modules}/tengine/modules/ngx_slab_stat \
--add-dynamic-module=${ngx_modules}/replace-filter-nginx-module \
--add-dynamic-module=${ngx_modules}/nginx-module-sysguard \
--add-dynamic-module=${ngx_modules}/delay-module \
--add-dynamic-module=${ngx_modules}/naxsi/naxsi_src \
--with-cc-opt="-I/usr/local/include -m64 -march=native -DTCP_FASTOPEN=23 -g -O3 -fstack-protector-strong -flto -fuse-ld=gold --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wno-deprecated-declarations -gsplit-dwarf" \
--with-ld-opt="-L/usr/local/lib -ljemalloc -Wl,-lpcre -Wl,-z,relro -Wl,-rpath,/usr/local/lib"
make && make test
make install
ldconfig
Check OpenResty version:
nginx -v
nginx version: openresty/1.15.8.1
And list all files in /etc/nginx
:
.
├── bin
│ ├── md2pod.pl
│ ├── nginx-xml2pod
│ ├── openresty -> /usr/sbin/nginx
│ ├── opm
│ ├── resty
│ ├── restydoc
│ └── restydoc-index
├── COPYRIGHT
├── fastcgi.conf
├── fastcgi.conf.default
├── fastcgi_params
├── fastcgi_params.default
├── koi-utf
├── koi-win
├── luajit
│ ├── bin
│ │ ├── luajit -> luajit-2.1.0-beta3
│ │ └── luajit-2.1.0-beta3
│ ├── include
│ │ └── luajit-2.1
│ │ ├── lauxlib.h
│ │ ├── luaconf.h
│ │ ├── lua.h
│ │ ├── lua.hpp
│ │ ├── luajit.h
│ │ └── lualib.h
│ ├── lib
│ │ ├── libluajit-5.1.a
│ │ ├── libluajit-5.1.so -> libluajit-5.1.so.2.1.0
│ │ ├── libluajit-5.1.so.2 -> libluajit-5.1.so.2.1.0
│ │ ├── libluajit-5.1.so.2.1.0
│ │ ├── lua
│ │ │ └── 5.1
│ │ └── pkgconfig
│ │ └── luajit.pc
│ └── share
│ ├── lua
│ │ └── 5.1
│ ├── luajit-2.1.0-beta3
│ │ └── jit
│ │ ├── bc.lua
│ │ ├── bcsave.lua
│ │ ├── dis_arm64be.lua
│ │ ├── dis_arm64.lua
│ │ ├── dis_arm.lua
│ │ ├── dis_mips64el.lua
│ │ ├── dis_mips64.lua
│ │ ├── dis_mipsel.lua
│ │ ├── dis_mips.lua
│ │ ├── dis_ppc.lua
│ │ ├── dis_x64.lua
│ │ ├── dis_x86.lua
│ │ ├── dump.lua
│ │ ├── p.lua
│ │ ├── v.lua
│ │ ├── vmdef.lua
│ │ └── zone.lua
│ └── man
│ └── man1
│ └── luajit.1
├── lualib
│ ├── cjson.so
│ ├── librestysignal.so
│ ├── ngx
│ │ ├── balancer.lua
│ │ ├── base64.lua
│ │ ├── errlog.lua
│ │ ├── ocsp.lua
│ │ ├── pipe.lua
│ │ ├── process.lua
│ │ ├── re.lua
│ │ ├── resp.lua
│ │ ├── semaphore.lua
│ │ ├── ssl
│ │ │ └── session.lua
│ │ └── ssl.lua
│ ├── resty
│ │ ├── aes.lua
│ │ ├── core
│ │ │ ├── base64.lua
│ │ │ ├── base.lua
│ │ │ ├── ctx.lua
│ │ │ ├── exit.lua
│ │ │ ├── hash.lua
│ │ │ ├── misc.lua
│ │ │ ├── ndk.lua
│ │ │ ├── phase.lua
│ │ │ ├── regex.lua
│ │ │ ├── request.lua
│ │ │ ├── response.lua
│ │ │ ├── shdict.lua
│ │ │ ├── time.lua
│ │ │ ├── uri.lua
│ │ │ ├── utils.lua
│ │ │ ├── var.lua
│ │ │ └── worker.lua
│ │ ├── core.lua
│ │ ├── dns
│ │ │ └── resolver.lua
│ │ ├── limit
│ │ │ ├── conn.lua
│ │ │ ├── count.lua
│ │ │ ├── req.lua
│ │ │ └── traffic.lua
│ │ ├── lock.lua
│ │ ├── lrucache
│ │ │ └── pureffi.lua
│ │ ├── lrucache.lua
│ │ ├── md5.lua
│ │ ├── random.lua
│ │ ├── sha1.lua
│ │ ├── sha224.lua
│ │ ├── sha256.lua
│ │ ├── sha384.lua
│ │ ├── sha512.lua
│ │ ├── sha.lua
│ │ ├── shell.lua
│ │ ├── signal.lua
│ │ ├── string.lua
│ │ ├── upload.lua
│ │ └── upstream
│ │ └── healthcheck.lua
│ └── tablepool.lua
├── mime.types
├── mime.types.default
├── modules
│ ├── ngx_http_delay_module.so
│ ├── ngx_http_naxsi_module.so
│ ├── ngx_http_replace_filter_module.so
│ └── ngx_http_sysguard_module.so
├── nginx
│ └── html
│ ├── 50x.html
│ └── index.html
├── nginx.conf
├── nginx.conf.default
├── pod
│ ├── array-var-nginx-module-0.05
│ │ └── array-var-nginx-module-0.05.pod
│ ├── drizzle-nginx-module-0.1.11
│ │ └── drizzle-nginx-module-0.1.11.pod
│ ├── echo-nginx-module-0.61
│ │ └── echo-nginx-module-0.61.pod
│ ├── encrypted-session-nginx-module-0.08
│ │ └── encrypted-session-nginx-module-0.08.pod
│ ├── form-input-nginx-module-0.12
│ │ └── form-input-nginx-module-0.12.pod
│ ├── headers-more-nginx-module-0.33
│ │ └── headers-more-nginx-module-0.33.pod
│ ├── iconv-nginx-module-0.14
│ │ └── iconv-nginx-module-0.14.pod
│ ├── lua-5.1.5
│ │ └── lua-5.1.5.pod
│ ├── lua-cjson-2.1.0.7
│ │ └── lua-cjson-2.1.0.7.pod
│ ├── luajit-2.1
│ │ ├── changes.pod
│ │ ├── contact.pod
│ │ ├── ext_c_api.pod
│ │ ├── extensions.pod
│ │ ├── ext_ffi_api.pod
│ │ ├── ext_ffi.pod
│ │ ├── ext_ffi_semantics.pod
│ │ ├── ext_ffi_tutorial.pod
│ │ ├── ext_jit.pod
│ │ ├── ext_profiler.pod
│ │ ├── faq.pod
│ │ ├── install.pod
│ │ ├── luajit-2.1.pod
│ │ ├── running.pod
│ │ └── status.pod
│ ├── luajit-2.1-20190507
│ │ └── luajit-2.1-20190507.pod
│ ├── lua-rds-parser-0.06
│ ├── lua-redis-parser-0.13
│ │ └── lua-redis-parser-0.13.pod
│ ├── lua-resty-core-0.1.17
│ │ ├── lua-resty-core-0.1.17.pod
│ │ ├── ngx.balancer.pod
│ │ ├── ngx.base64.pod
│ │ ├── ngx.errlog.pod
│ │ ├── ngx.ocsp.pod
│ │ ├── ngx.pipe.pod
│ │ ├── ngx.process.pod
│ │ ├── ngx.re.pod
│ │ ├── ngx.resp.pod
│ │ ├── ngx.semaphore.pod
│ │ ├── ngx.ssl.pod
│ │ └── ngx.ssl.session.pod
│ ├── lua-resty-dns-0.21
│ │ └── lua-resty-dns-0.21.pod
│ ├── lua-resty-limit-traffic-0.06
│ │ ├── lua-resty-limit-traffic-0.06.pod
│ │ ├── resty.limit.conn.pod
│ │ ├── resty.limit.count.pod
│ │ ├── resty.limit.req.pod
│ │ └── resty.limit.traffic.pod
│ ├── lua-resty-lock-0.08
│ │ └── lua-resty-lock-0.08.pod
│ ├── lua-resty-lrucache-0.09
│ │ └── lua-resty-lrucache-0.09.pod
│ ├── lua-resty-memcached-0.14
│ │ └── lua-resty-memcached-0.14.pod
│ ├── lua-resty-mysql-0.21
│ │ └── lua-resty-mysql-0.21.pod
│ ├── lua-resty-redis-0.27
│ │ └── lua-resty-redis-0.27.pod
│ ├── lua-resty-shell-0.02
│ │ └── lua-resty-shell-0.02.pod
│ ├── lua-resty-signal-0.02
│ │ └── lua-resty-signal-0.02.pod
│ ├── lua-resty-string-0.11
│ │ └── lua-resty-string-0.11.pod
│ ├── lua-resty-upload-0.10
│ │ └── lua-resty-upload-0.10.pod
│ ├── lua-resty-upstream-healthcheck-0.06
│ │ └── lua-resty-upstream-healthcheck-0.06.pod
│ ├── lua-resty-websocket-0.07
│ │ └── lua-resty-websocket-0.07.pod
│ ├── lua-tablepool-0.01
│ │ └── lua-tablepool-0.01.pod
│ ├── memc-nginx-module-0.19
│ │ └── memc-nginx-module-0.19.pod
│ ├── nginx
│ │ ├── accept_failed.pod
│ │ ├── beginners_guide.pod
│ │ ├── chunked_encoding_from_backend.pod
│ │ ├── configure.pod
│ │ ├── configuring_https_servers.pod
│ │ ├── contributing_changes.pod
│ │ ├── control.pod
│ │ ├── converting_rewrite_rules.pod
│ │ ├── daemon_master_process_off.pod
│ │ ├── debugging_log.pod
│ │ ├── development_guide.pod
│ │ ├── events.pod
│ │ ├── example.pod
│ │ ├── faq.pod
│ │ ├── freebsd_tuning.pod
│ │ ├── hash.pod
│ │ ├── howto_build_on_win32.pod
│ │ ├── install.pod
│ │ ├── license_copyright.pod
│ │ ├── load_balancing.pod
│ │ ├── nginx_dtrace_pid_provider.pod
│ │ ├── nginx.pod
│ │ ├── ngx_core_module.pod
│ │ ├── ngx_google_perftools_module.pod
│ │ ├── ngx_http_access_module.pod
│ │ ├── ngx_http_addition_module.pod
│ │ ├── ngx_http_api_module_head.pod
│ │ ├── ngx_http_auth_basic_module.pod
│ │ ├── ngx_http_auth_jwt_module.pod
│ │ ├── ngx_http_auth_request_module.pod
│ │ ├── ngx_http_autoindex_module.pod
│ │ ├── ngx_http_browser_module.pod
│ │ ├── ngx_http_charset_module.pod
│ │ ├── ngx_http_core_module.pod
│ │ ├── ngx_http_dav_module.pod
│ │ ├── ngx_http_empty_gif_module.pod
│ │ ├── ngx_http_f4f_module.pod
│ │ ├── ngx_http_fastcgi_module.pod
│ │ ├── ngx_http_flv_module.pod
│ │ ├── ngx_http_geoip_module.pod
│ │ ├── ngx_http_geo_module.pod
│ │ ├── ngx_http_grpc_module.pod
│ │ ├── ngx_http_gunzip_module.pod
│ │ ├── ngx_http_gzip_module.pod
│ │ ├── ngx_http_gzip_static_module.pod
│ │ ├── ngx_http_headers_module.pod
│ │ ├── ngx_http_hls_module.pod
│ │ ├── ngx_http_image_filter_module.pod
│ │ ├── ngx_http_index_module.pod
│ │ ├── ngx_http_js_module.pod
│ │ ├── ngx_http_keyval_module.pod
│ │ ├── ngx_http_limit_conn_module.pod
│ │ ├── ngx_http_limit_req_module.pod
│ │ ├── ngx_http_log_module.pod
│ │ ├── ngx_http_map_module.pod
│ │ ├── ngx_http_memcached_module.pod
│ │ ├── ngx_http_mirror_module.pod
│ │ ├── ngx_http_mp4_module.pod
│ │ ├── ngx_http_perl_module.pod
│ │ ├── ngx_http_proxy_module.pod
│ │ ├── ngx_http_random_index_module.pod
│ │ ├── ngx_http_realip_module.pod
│ │ ├── ngx_http_referer_module.pod
│ │ ├── ngx_http_rewrite_module.pod
│ │ ├── ngx_http_scgi_module.pod
│ │ ├── ngx_http_secure_link_module.pod
│ │ ├── ngx_http_session_log_module.pod
│ │ ├── ngx_http_slice_module.pod
│ │ ├── ngx_http_spdy_module.pod
│ │ ├── ngx_http_split_clients_module.pod
│ │ ├── ngx_http_ssi_module.pod
│ │ ├── ngx_http_ssl_module.pod
│ │ ├── ngx_http_status_module.pod
│ │ ├── ngx_http_stub_status_module.pod
│ │ ├── ngx_http_sub_module.pod
│ │ ├── ngx_http_upstream_conf_module.pod
│ │ ├── ngx_http_upstream_hc_module.pod
│ │ ├── ngx_http_upstream_module.pod
│ │ ├── ngx_http_userid_module.pod
│ │ ├── ngx_http_uwsgi_module.pod
│ │ ├── ngx_http_v2_module.pod
│ │ ├── ngx_http_xslt_module.pod
│ │ ├── ngx_mail_auth_http_module.pod
│ │ ├── ngx_mail_core_module.pod
│ │ ├── ngx_mail_imap_module.pod
│ │ ├── ngx_mail_pop3_module.pod
│ │ ├── ngx_mail_proxy_module.pod
│ │ ├── ngx_mail_smtp_module.pod
│ │ ├── ngx_mail_ssl_module.pod
│ │ ├── ngx_stream_access_module.pod
│ │ ├── ngx_stream_core_module.pod
│ │ ├── ngx_stream_geoip_module.pod
│ │ ├── ngx_stream_geo_module.pod
│ │ ├── ngx_stream_js_module.pod
│ │ ├── ngx_stream_keyval_module.pod
│ │ ├── ngx_stream_limit_conn_module.pod
│ │ ├── ngx_stream_log_module.pod
│ │ ├── ngx_stream_map_module.pod
│ │ ├── ngx_stream_proxy_module.pod
│ │ ├── ngx_stream_realip_module.pod
│ │ ├── ngx_stream_return_module.pod
│ │ ├── ngx_stream_split_clients_module.pod
│ │ ├── ngx_stream_ssl_module.pod
│ │ ├── ngx_stream_ssl_preread_module.pod
│ │ ├── ngx_stream_upstream_hc_module.pod
│ │ ├── ngx_stream_upstream_module.pod
│ │ ├── ngx_stream_zone_sync_module.pod
│ │ ├── request_processing.pod
│ │ ├── server_names.pod
│ │ ├── stream_processing.pod
│ │ ├── switches.pod
│ │ ├── syntax.pod
│ │ ├── sys_errlist.pod
│ │ ├── syslog.pod
│ │ ├── variables_in_config.pod
│ │ ├── websocket.pod
│ │ ├── welcome_nginx_facebook.pod
│ │ └── windows.pod
│ ├── ngx_coolkit-0.2
│ ├── ngx_devel_kit-0.3.1rc1
│ │ └── ngx_devel_kit-0.3.1rc1.pod
│ ├── ngx_lua-0.10.15
│ │ └── ngx_lua-0.10.15.pod
│ ├── ngx_lua_upstream-0.07
│ │ └── ngx_lua_upstream-0.07.pod
│ ├── ngx_postgres-1.0
│ │ ├── ngx_postgres-1.0.pod
│ │ └── todo.pod
│ ├── ngx_stream_lua-0.0.7
│ │ ├── dev_notes.pod
│ │ └── ngx_stream_lua-0.0.7.pod
│ ├── opm-0.0.5
│ │ └── opm-0.0.5.pod
│ ├── rds-csv-nginx-module-0.09
│ │ └── rds-csv-nginx-module-0.09.pod
│ ├── rds-json-nginx-module-0.15
│ │ └── rds-json-nginx-module-0.15.pod
│ ├── redis2-nginx-module-0.15
│ │ └── redis2-nginx-module-0.15.pod
│ ├── redis-nginx-module-0.3.7
│ ├── resty-cli-0.24
│ │ └── resty-cli-0.24.pod
│ ├── set-misc-nginx-module-0.32
│ │ └── set-misc-nginx-module-0.32.pod
│ ├── srcache-nginx-module-0.31
│ │ └── srcache-nginx-module-0.31.pod
│ └── xss-nginx-module-0.06
│ └── xss-nginx-module-0.06.pod
├── resty.index
├── scgi_params
├── scgi_params.default
├── site
│ ├── lualib
│ ├── manifest
│ └── pod
├── uwsgi_params
├── uwsgi_params.default
└── win-utf
78 directories, 305 files
Check all post installation tasks from Nginx on CentOS 7 - Post installation tasks section.
Tengine is a web server originated by Taobao, the largest e-commerce website in Asia. It is based on the NGINX HTTP server and has many advanced features. There’s a lot of features in Tengine that do not (yet) exist in NGINX.
Generally, Tengine is a great solution, including many patches, improvements, additional modules, and most importantly it is very actively maintained.
The build and installation process is very similar to Install Nginx on Centos 7. However, I will only specify the most important changes.
Set the Tengine version (I use newest and stable release):
export ngx_version="2.3.0"
Set temporary variables:
ngx_src="../../usr/local/src"
ngx_base="${ngx_src}/tengine-${ngx_version}"
ngx_master="${ngx_base}/master"
ngx_modules="${ngx_base}/modules"
Create directories:
for i in "$ngx_base" "${ngx_master}" "$ngx_modules" ; do
mkdir "$i"
done
Install prebuilt packages, export variables and set symbolic link:
apt-get install gcc make build-essential bison perl libperl-dev libphp-embed libxslt-dev libgd-dev libgeoip-dev libxml2-dev libexpat-dev libgoogle-perftools-dev libgoogle-perftools4 autoconf jq
# In this example we don't use zlib sources:
apt-get install zlib1g-dev
PCRE:
cd "${ngx_src}"
export pcre_version="8.42"
export PCRE_SRC="${ngx_base}/pcre-${pcre_version}"
export PCRE_LIB="../../usr/local/lib"
export PCRE_INC="../../usr/local/include"
wget https://ftp.pcre.org/pub/pcre/pcre-${pcre_version}.tar.gz && tar xzvf pcre-${pcre_version}.tar.gz
cd "$PCRE_SRC"
./configure
make -j2 && make test
make install
OpenSSL:
cd "${ngx_src}"
export openssl_version="1.1.1b"
export OPENSSL_SRC="${ngx_src}/openssl-${openssl_version}"
export OPENSSL_DIR="../../usr/local/openssl-${openssl_version}"
export OPENSSL_LIB="${OPENSSL_DIR}/lib"
export OPENSSL_INC="${OPENSSL_DIR}/include"
wget https://www.openssl.org/source/openssl-${openssl_version}.tar.gz && tar xzvf openssl-${openssl_version}.tar.gz
cd "${ngx_src}/openssl-${openssl_version}"
# Please run this and add as a compiler param:
export __GCC_SSL=("__SIZEOF_INT128__:enable-ec_nistp_64_gcc_128")
for _cc_opt in "${__GCC_SSL[@]}" ; do
_cc_key=$(echo "$_cc_opt" | cut -d ":" -f1)
_cc_value=$(echo "$_cc_opt" | cut -d ":" -f2)
if [[ ! $(gcc -dM -E - </dev/null | grep -q "$_cc_key") ]] ; then
echo -en "$_cc_value is supported on this machine\n"
_openssl_gcc+="$_cc_value "
fi
done
./config --prefix="$OPENSSL_DIR" --openssldir="$OPENSSL_DIR" shared zlib no-ssl3 no-weak-ssl-ciphers -DOPENSSL_NO_HEARTBEATS -fstack-protector-strong "$_openssl_gcc"
make -j2 && make test
make install
# Setup PATH environment variables:
cat > /etc/profile.d/openssl.sh << __EOF__
#!/bin/sh
export PATH=${OPENSSL_DIR}/bin:${PATH}
export LD_LIBRARY_PATH=${OPENSSL_DIR}/lib:${LD_LIBRARY_PATH}
__EOF__
chmod +x /etc/profile.d/openssl.sh && source /etc/profile.d/openssl.sh
# To make the OpenSSL 1.1.1b version visible globally first:
mv /usr/bin/openssl /usr/bin/openssl-old
ln -s ${OPENSSL_DIR}/bin/openssl /usr/bin/openssl
cat > /etc/ld.so.conf.d/openssl.conf << __EOF__
${OPENSSL_DIR}/lib
__EOF__
LuaJIT:
# I recommend to use OpenResty's branch (openresty/luajit2) instead LuaJIT (LuaJIT/LuaJIT), but both installation methods are similar:
cd "${ngx_src}"
export LUAJIT_SRC="${ngx_src}/luajit2"
export LUAJIT_LIB="../../usr/local/lib"
export LUAJIT_INC="../../usr/local/include/luajit-2.1"
# For originall LuaJIT:
# git clone http://luajit.org/git/luajit-2.0 luajit2
# cd "$LUAJIT_SRC"
# For OpenResty's LuaJIT:
git clone --depth 1 https://github.com/openresty/luajit2
cd "$LUAJIT_SRC"
make && make install
ln -s /usr/local/lib/libluajit-5.1.so.2.1.0 /usr/local/lib/liblua.so
sregex:
Required for
replace-filter-nginx-module
module.
cd "${ngx_src}"
git clone --depth 1 https://github.com/openresty/sregex
cd "${ngx_src}/sregex"
make && make install
jemalloc:
To verify
jemalloc
in use:lsof -n | grep jemalloc
.
cd "${ngx_src}"
export JEMALLOC_SRC="../../usr/local/src/jemalloc"
export JEMALLOC_INC="../../usr/local/include/jemalloc"
git clone --depth 1 https://github.com/jemalloc/jemalloc
cd "$JEMALLOC_SRC"
./autogen.sh
make && make install
Update links and cache to the shared libraries for both types of installation:
ldconfig
cd "${ngx_base}"
wget https://tengine.taobao.org/download/tengine-${ngx_version}.tar.gz
# or alternative:
# git clone --depth 1 https://github.com/alibaba/tengine master
tar zxvf tengine-${ngx_version}.tar.gz -C "${ngx_master}"
Not all modules from this section working properly with Tengine (e.g.
ndk_http_module
and other dependent on it).
cd "${ngx_modules}"
for i in \
https://github.com/openresty/echo-nginx-module \
https://github.com/openresty/headers-more-nginx-module \
https://github.com/openresty/replace-filter-nginx-module \
https://github.com/nginx-clojure/nginx-access-plus \
https://github.com/yaoweibin/ngx_http_substitutions_filter_module \
https://github.com/vozlt/nginx-module-vts \
https://github.com/google/ngx_brotli ; do
git clone --depth 1 "$i"
done
wget http://mdounin.ru/hg/ngx_http_delay_module/archive/tip.tar.gz -O delay-module.tar.gz
mkdir delay-module && tar xzvf delay-module.tar.gz -C delay-module --strip 1
For ngx_brotli
:
cd "${ngx_modules}/ngx_brotli"
git submodule update --init
If you use NAXSI:
cd "${ngx_modules}"
git clone --depth 1 https://github.com/nbs-system/naxsi
cd "${ngx_master}"
# - you can also build Tengine without 3rd party modules
# - remember about compiler and linker options
# - don't set values for --with-openssl, --with-pcre, and --with-zlib if you select prebuilt packages for them
./configure --prefix=/etc/nginx \
--conf-path=/etc/nginx/nginx.conf \
--sbin-path=/usr/sbin/nginx \
--pid-path=/var/run/nginx.pid \
--lock-path=/var/run/nginx.lock \
--user=nginx \
--group=nginx \
--modules-path=/etc/nginx/modules \
--error-log-path=/var/log/nginx/error.log \
--http-log-path=/var/log/nginx/access.log \
--http-client-body-temp-path=/var/cache/nginx/client_temp \
--http-proxy-temp-path=/var/cache/nginx/proxy_temp \
--http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp \
--http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp \
--http-scgi-temp-path=/var/cache/nginx/scgi_temp \
--with-compat \
--with-debug \
--with-file-aio \
--with-threads \
--with-stream \
--with-stream_geoip_module \
--with-stream_realip_module \
--with-stream_ssl_module \
--with-stream_ssl_preread_module \
--with-http_addition_module \
--with-http_auth_request_module \
--with-http_degradation_module \
--with-http_geoip_module \
--with-http_gunzip_module \
--with-http_gzip_static_module \
--with-http_image_filter_module \
--with-http_lua_module \
--with-http_perl_module \
--with-http_random_index_module \
--with-http_realip_module \
--with-http_secure_link_module \
--with-http_ssl_module \
--with-http_stub_status_module \
--with-http_sub_module \
--with-http_v2_module \
--with-google_perftools_module \
--with-openssl=${OPENSSL_SRC} \
--with-openssl-opt="shared zlib no-ssl3 no-weak-ssl-ciphers -DOPENSSL_NO_HEARTBEATS -fstack-protector-strong ${_openssl_gcc}" \
--with-pcre=${PCRE_SRC} \
--with-pcre-jit \
--with-jemalloc=${JEMALLOC_SRC} \
--without-http-cache \
--without-http_memcached_module \
--without-mail_pop3_module \
--without-mail_imap_module \
--without-mail_smtp_module \
--without-http_fastcgi_module \
--without-http_scgi_module \
--without-http_uwsgi_module \
--without-http_upstream_keepalive_module \
--add-module=${ngx_master}/modules/ngx_backtrace_module \
--add-module=${ngx_master}/modules/ngx_debug_pool \
--add-module=${ngx_master}/modules/ngx_debug_timer \
--add-module=${ngx_master}/modules/ngx_http_footer_filter_module \
--add-module=${ngx_master}/modules/ngx_http_lua_module \
--add-module=${ngx_master}/modules/ngx_http_proxy_connect_module \
--add-module=${ngx_master}/modules/ngx_http_reqstat_module \
--add-module=${ngx_master}/modules/ngx_http_slice_module \
--add-module=${ngx_master}/modules/ngx_http_sysguard_module \
--add-module=${ngx_master}/modules/ngx_http_trim_filter_module \
--add-module=${ngx_master}/modules/ngx_http_upstream_check_module \
--add-module=${ngx_master}/modules/ngx_http_upstream_consistent_hash_module \
--add-module=${ngx_master}/modules/ngx_http_upstream_dynamic_module \
--add-module=${ngx_master}/modules/ngx_http_upstream_keepalive_module \
--add-module=${ngx_master}/modules/ngx_http_upstream_session_sticky_module \
--add-module=${ngx_master}/modules/ngx_http_user_agent_module \
--add-module=${ngx_master}/modules/ngx_slab_stat \
--add-module=${ngx_modules}/nginx-access-plus/src/c \
--add-module=${ngx_modules}/ngx_http_substitutions_filter_module \
--add-module=${ngx_modules}/nginx-module-vts \
--add-module=${ngx_modules}/ngx_brotli \
--add-dynamic-module=${ngx_modules}/echo-nginx-module \
--add-dynamic-module=${ngx_modules}/headers-more-nginx-module \
--add-dynamic-module=${ngx_modules}/replace-filter-nginx-module \
--add-dynamic-module=${ngx_modules}/delay-module \
--add-dynamic-module=${ngx_modules}/naxsi/naxsi_src \
--with-cc-opt="-I/usr/local/include -I${OPENSSL_INC} -I${LUAJIT_INC} -I${JEMALLOC_INC} -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC" \
--with-ld-opt="-Wl,-E -L/usr/local/lib -ljemalloc -lpcre -Wl,-rpath,/usr/local/lib/,-z,relro -Wl,-z,now -pie"
make -j2 && make test
make install
ldconfig
Check Tengine version:
nginx -v
Tengine version: Tengine/2.3.0
nginx version: nginx/1.15.9
And list all files in /etc/nginx
:
tree
.
├── fastcgi.conf
├── fastcgi.conf.default
├── fastcgi_params
├── fastcgi_params.default
├── html
│ ├── 50x.html
│ └── index.html
├── koi-utf
├── koi-win
├── mime.types
├── mime.types.default
├── modules
│ ├── ngx_http_delay_module.so
│ ├── ngx_http_echo_module.so
│ ├── ngx_http_headers_more_filter_module.so
│ ├── ngx_http_naxsi_module.so
│ └── ngx_http_replace_filter_module.so
├── nginx.conf
├── nginx.conf.default
├── scgi_params
├── scgi_params.default
├── uwsgi_params
├── uwsgi_params.default
└── win-utf
2 directories, 22 files
Check all post installation tasks from Nginx on CentOS 7 - Post installation tasks section.
These are the basic set of rules to keep NGINX in good condition.
When your NGINX configuration grow, the need for organising your configuration will also grow. Well organised code is:
- easier to understand
- easier to maintain
- easier to work with
Use
include
directive to move common server settings into a separate files and to attach your NGINX specific code to global config, contexts and other.
# Store this configuration in e.g. https-ssl-common.conf
listen 10.240.20.2:443 ssl;
root /etc/nginx/error-pages/other;
ssl_certificate /etc/nginx/domain.com/certs/nginx_domain.com_bundle.crt;
ssl_certificate_key /etc/nginx/domain.com/certs/domain.com.key;
# And include this file in server section:
server {
include /etc/nginx/domain.com/commons/https-ssl-common.conf;
server_name domain.com www.domain.com;
...
Work with unreadable configuration files is terrible, if syntax isn’t very readable, it makes your eyes sore, and you suffers from headaches.
When your code is formatted, it is significantly easier to maintain, debug, optimize, and can be read and understood in a short amount of time. You should eliminate code style violations from your NGINX configuration files.
Choose your formatter style and setup a common config for it. Some rules are universal, but the most important thing is to keep a consistent NGINX code style throughout your code base:
- use whitespaces and blank lines to arrange and separate code blocks
- use tabs for indents - they are consistent, customizable and allow mistakes to be more noticeable (unless you are a 4 space kind of guy)
- use comments to explain why things are done not what is done
- use meaningful naming conventions
- simple is better than complex but complex is better than complicated
Of course, the NGINX configuration code is not a programming language. All files are written in their own language or syntax so we should not overdo it, but I think it's worth sticking to the general rules and make your and other NGINX adminstrators life easier.
# Good NGINX code style:
http {
# Attach global rules:
include /etc/nginx/proxy.conf;
include /etc/nginx/fastcgi.conf;
index index.html index.htm index.php;
default_type application/octet-stream;
# Standard log format:
log_format main '$remote_addr - $remote_user [$time_local] $status '
'"$request" $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
# This seems to be required for some vhosts:
server_names_hash_bucket_size 128;
...
# Bad NGINX code style:
http {
include nginx/proxy.conf;
include /etc/nginx/fastcgi.conf;
index index.html index.htm index.php;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] $status '
'"$request" $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log logs/access.log main;
sendfile on;
tcp_nopush on;
server_names_hash_bucket_size 128; # this seems to be required for some vhosts
...
reload
method to change configurations on the flyUse the
reload
method of NGINX to achieve a graceful reload of the configuration without stopping the server and dropping any packets. This function of the master process allows to rolls back the changes and continues to work with stable and old working configuration.
This ability of NGINX is very critical in a high-uptime, dynamic environments for keeping the load balancer or standalone server online.
Master process checks the syntax validity of the new configuration and tries to apply all changes. If this procedure has been accomplished, the master process create new worker processes and sends shutdown messages to old. Old workers stops accepting new connections after received a shut down signal but current requests are still processing. After that, the old workers exit.
When you restart NGINX you might encounter situation in which NGINX will stop, and won't start back again, because of syntax error. Reload method is safer than restarting because before old process will be terminated, new configuration file is parsed and whole process is aborted if there are any problems with it.
To stop NGINX processes with waiting for the worker processes to finish serving current requests use
nginx -s quit
command. It's better thannginx -s stop
for fast shutdown.
From NGINX documentation:
In order for NGINX to re-read the configuration file, a HUP signal should be sent to the master process. The master process first checks the syntax validity, then tries to apply new configuration, that is, to open log files and new listen sockets. If this fails, it rolls back changes and continues to work with old configuration. If this succeeds, it starts new worker processes, and sends messages to old worker processes requesting them to shut down gracefully. Old worker processes close listen sockets and continue to service old clients. After all clients are serviced, old worker processes are shut down.
# 1)
systemctl reload nginx
# 2)
service nginx reload
# 3)
/etc/init.d/nginx reload
# 4)
/usr/sbin/nginx -s reload
# 5)
kill -HUP $(cat /var/run/nginx.pid)
# or
kill -HUP $(ps auxw | grep [n]ginx | grep master | awk '{print $2}')
listen
directives for 80 and 443I don't like duplicating the rules, but it's certainly an easy and maintainable way.
# For http:
server {
listen 10.240.20.2:80;
...
}
# For https:
server {
listen 10.240.20.2:443 ssl;
...
}
listen
directives explicitly with address:port
pairNGINX translates all incomplete
listen
directives by substituting missing values with their default values.
NGINX will only evaluate the
server_name
directive when it needs to distinguish between server blocks that match to the same level in the listen directive.
Set IP address and port number to prevents soft mistakes which may be difficult to debug.
server {
# This block will be processed:
listen 192.168.252.10; # --> 192.168.252.10:80
...
}
server {
listen 80; # --> *:80 --> 0.0.0.0:80
server_name api.random.com;
...
}
NGINX should prevent processing requests with undefined server names (also on IP address). It also protects against configuration errors and don't pass traffic to incorrect backends. The problem is easily solved by creating a default catch all server config.
If none of the listen directives have the
default_server
parameter then the first server with theaddress:port
pair will be the default server for this pair (it means that NGINX always has a default server).
If someone makes a request using an IP address instead of a server name, the
Host
request header field will contain the IP address and the request can be handled using the IP address as the server name.
Also good point is
return 444;
for default server name because this will close the connection and log it internally, for any domain that isn't defined in NGINX.
# Place it at the beginning of the configuration file to prevent mistakes.
server {
# Add default_server to your listen directive in the server that you want to act as the default.
listen 10.240.20.2:443 default_server ssl;
# We catch:
# - invalid domain names
# - requests without the "Host" header
# - and all others (also due to the above setting)
# - default_server in server_name directive is not required - I add this for a better understanding
server_name _ "" default_server;
...
return 444;
# We can also serve:
# location / {
# static file (error page):
# root /etc/nginx/error-pages/404;
# or redirect:
# return 301 https://badssl.com;
# return 444;
# }
}
server {
listen 10.240.20.2:443 ssl;
server_name domain.com;
...
}
server {
listen 10.240.20.2:443 ssl;
server_name domain.org;
...
}
For sharing a single IP address between several HTTPS servers you should use one SSL config (e.g. protocols, ciphers, curves) because changes will affect only the default server.
Remember that regardless of SSL parameters, you are able to use multiple SSL certificates.
If you want to set up different SSL configurations for the same IP address then it will fail. It's important because SSL configuration is presented for default server - if none of the listen directives have the
default_server
parameter then the first server in your configuration. So you should use only one SSL setup with several names on the same IP address.
It's also to prevent mistakes and configuration mismatch.
# Store this configuration in e.g. https.conf
listen 192.168.252.10:443 default_server ssl http2;
ssl_protocols TLSv1.2;
ssl_ciphers "ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384";
ssl_prefer_server_ciphers on;
ssl_ecdh_curve secp521r1:secp384r1;
...
# Include this file to the server context (attach domain-a.com for specific listen directive)
server {
include /etc/nginx/https.conf;
server_name domain-a.com;
...
}
# Include this file to the server context (attach domain-b.com for specific listen directive)
server {
include /etc/nginx/https.conf;
server_name domain-b.com;
...
}
You should always use HTTPS instead of HTTP to protect your website, even if it doesn’t handle sensitive communications.
We have currently the first free and open CA - Let's Encrypt - so generating and implementing certificates has never been so easy. It was created to provide free and easy-to-use TLS and SSL certificates.
server {
listen 10.240.20.2:80;
server_name domain.com;
return 301 https://$host$request_uri;
}
server {
listen 10.240.20.2:443 ssl;
server_name domain.com;
...
}
Use map or geo modules (one of them) to prevent users abusing your servers.
This allows to create variables with values depending on the client IP address.
# Map module:
map $remote_addr $globals_internal_map_acl {
# Status code:
# - 0 = false
# - 1 = true
default 0;
### INTERNAL ###
10.255.10.0/24 1;
10.255.20.0/24 1;
10.255.30.0/24 1;
192.168.0.0/16 1;
}
# Geo module:
geo $globals_internal_geo_acl {
# Status code:
# - 0 = false
# - 1 = true
default 0;
### INTERNAL ###
10.255.10.0/24 1;
10.255.20.0/24 1;
10.255.30.0/24 1;
192.168.0.0/16 1;
}
Manage a large number of redirects with NGINX maps and use them to customize your key-value pairs.
Map module provides a more elegant solution for clearly parsing a big list of regexes, e.g. User-Agents, Referrers.
map $http_user_agent $device_redirect {
default "desktop";
~(?i)ip(hone|od) "mobile";
~(?i)android.*(mobile|mini) "mobile";
~Mobile.+Firefox "mobile";
~^HTC "mobile";
~Fennec "mobile";
~IEMobile "mobile";
~BB10 "mobile";
~SymbianOS.*AppleWebKit "mobile";
~Opera\sMobi "mobile";
}
# Turn on in a specific context (e.g. location):
if ($device_redirect = "mobile") {
return 301 https://m.domain.com$request_uri;
}
If you add a root to every location block then a location block that isn’t matched will have no root. Set global
root
inside server directive.
server {
server_name domain.com;
root /var/www/domain.com/public;
location / {
...
}
location /api {
...
}
location /static {
root /var/www/domain.com/static;
...
}
}
There's probably more detail than you want, but that can sometimes be a lifesaver (but log file growing rapidly on a very high-traffic sites).
rewrite_log on;
error_log /var/log/nginx/error-debug.log debug;
Anything you can access as a variable in NGINX config, you can log, including non-standard http headers, etc. so it's a simple way to create your own log format for specific situations.
This is extremely helpful for debugging specific
location
directives.
# Default main log format from NGINX repository:
log_format main
'$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
# Extended main log format:
log_format main-level-0
'$remote_addr - $remote_user [$time_local] '
'"$request_method $scheme://$host$request_uri '
'$server_protocol" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent" '
'$request_time';
# Debug log formats:
log_format debug-level-0
'$remote_addr - $remote_user [$time_local] '
'"$request_method $scheme://$host$request_uri '
'$server_protocol" $status $body_bytes_sent '
'$request_id $pid $msec $request_time '
'$upstream_connect_time $upstream_header_time '
'$upstream_response_time "$request_filename" '
'$request_completion';
log_format debug-level-1
'$remote_addr - $remote_user [$time_local] '
'"$request_method $scheme://$host$request_uri '
'$server_protocol" $status $body_bytes_sent '
'$request_id $pid $msec $request_time '
'$upstream_connect_time $upstream_header_time '
'$upstream_response_time "$request_filename" $request_length '
'$request_completion $connection $connection_requests '
'"$http_user_agent"';
log_format debug-level-2
'$remote_addr - $remote_user [$time_local] '
'"$request_method $scheme://$host$request_uri '
'$server_protocol" $status $body_bytes_sent '
'$request_id $pid $msec $request_time '
'$upstream_connect_time $upstream_header_time '
'$upstream_response_time "$request_filename" $request_length '
'$request_completion $connection $connection_requests '
'$remote_addr $remote_port $server_addr $server_port '
'$http_x_forwarded_for "$http_referer" "$http_user_agent"';
# Debug log format for SSL:
log_format debug-ssl-level-0
'$remote_addr - $remote_user [$time_local] '
'"$request_method $scheme://$host$request_uri '
'$server_protocol" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent" '
'$request_time '
'$tls_version $ssl_protocol $ssl_cipher';
NGINX is a insanely fast, but you can adjust a few things to make sure it's as fast as possible for your use case.
The
worker_processes
directive is the sturdy spine of life for NGINX. This directive is responsible for letting our virtual server know many workers to spawn once it has become bound to the proper IP and port(s).
I think for high load proxy servers (also standalone servers) good value is
ALL_CORES - 1
(please test it before used).
Rule of thumb: If much time is spent blocked on I/O, worker processes should be increased further.
Official NGINX documentation say:
When one is in doubt, setting it to the number of available CPU cores would be a good start (the value "auto" will try to autodetect it).
# VCPU = 4 , expr $(nproc --all) - 1
worker_processes 3;
The primary goals for HTTP/2 are to reduce latency by enabling full request and response multiplexing, minimize protocol overhead via efficient compression of HTTP header fields, and add support for request prioritization and server push.
HTTP/2 will make our applications faster, simpler, and more robust.
HTTP/2 is backwards-compatible with HTTP/1.1, so it would be possible to ignore it completely and everything will continue to work as before because if the client that does not support HTTP/2 will never ask the server for an HTTP/2 communication upgrade: the communication between them will be fully HTTP1/1.
Also include the
ssl
parameter, required because browsers do not support HTTP/2 without encryption.
HTTP/2 has a extremely large blacklist of old and insecure ciphers, so you should avoid them.
# For https:
server {
listen 10.240.20.2:443 ssl http2;
...
This improves performance from the clients’ perspective, because it eliminates the need for a new (and time-consuming) SSL handshake to be conducted each time a request is made.
Most servers do not purge sessions or ticket keys, thus increasing the risk that a server compromise would leak data from previous (and future) connections.
Set SSL Session Timeout to
5
minutes for prevent abused by advertisers like Google and Facebook.
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 5m;
ssl_session_tickets off;
ssl_buffer_size 1400;
server_name
directive where possibleExact names, wildcard names starting with an asterisk, and wildcard names ending with an asterisk are stored in three hash tables bound to the listen ports.
The exact names hash table is searched first. If a name is not found, the hash table with wildcard names starting with an asterisk is searched. If the name is not found there, the hash table with wildcard names ending with an asterisk is searched. Searching wildcard names hash table is slower than searching exact names hash table because names are searched by domain parts.
Regular expressions are tested sequentially and therefore are the slowest method and are non-scalable. For these reasons, it is better to use exact names where possible.
# It is more efficient to define them explicitly:
server {
listen 80;
server_name example.org www.example.org *.example.org;
...
}
# than to use the simplified form:
server {
listen 80;
server_name .example.org;
...
}
server_name
with if
directiveWhen NGINX receives a request no matter what is the subdomain being requested, be it
www.example.com
or just the plainexample.com
thisif
directive is always evaluated. Since you’re requesting NGINX to check for theHost
header for every request. It’s extremely inefficient.
Instead use two server directives like the example below. This approach decreases NGINX processing requirements.
# Bad configuration:
server {
...
server_name domain.com www.domain.com;
if ($host = www.domain.com) {
return 301 https://domain.com$request_uri;
}
server_name domain.com;
...
}
# Good configuration:
server {
server_name www.domain.com;
return 301 $scheme://domain.com$request_uri;
# If you force your web traffic to use HTTPS:
# 301 https://domain.com$request_uri;
}
server {
listen 80;
server_name domain.com;
...
}
Exact location matches are often used to speed up the selection process by immediately ending the execution of the algorithm.
# Matches the query / only and stops searching:
location = / {
...
}
# Matches the query /v9 only and stops searching:
location = /v9 {
...
}
...
# Matches any query due to the fact that all queries begin at /,
# but regular expressions and any longer conventional blocks will be matched at first place:
location / {
...
}
limit_conn
to improve limiting the download speedNGINX provides two directives to limiting download speed:
limit_rate_after
- sets the amount of data transferred before thelimit_rate
directive takes effectlimit_rate
- allows you to limit the transfer rate of individual client connections (past exceedinglimit_rate_after
)
This solution limits NGINX download speed per connection, so, if one user opens multiple e.g. video files, it will be able to download
X * the number of times
he connected to the video files.
To prevent this situation use
limit_conn_zone
andlimit_conn
directives.
# Create limit connection zone:
limit_conn_zone $binary_remote_addr zone=conn_for_remote_addr:1m;
# Add rules to limiting the download speed:
limit_rate_after 1m; # run at maximum speed for the first 1 megabyte
limit_rate 250k; # and set rate limit after 1 megabyte
# Enable queue:
location /videos {
# Max amount of data by one client: 10 megabytes (limit_rate_after * 10)
limit_conn conn_for_remote_addr 10;
...
In this chapter I will talk about some of the NGINX hardening approaches and security standards.
NGINX is a very secure and stable but vulnerabilities in the main binary itself do pop up from time to time. It's the main reason for keep NGINX up-to-date as hard as you can.
A very safe way to plan the update is once a new stable version is released but for me the most common way to handle NGINX updates is to wait a few weeks after the stable release.
Before update/upgrade NGINX remember about do it on the testing environment.
Most modern GNU/Linux distros will not push the latest version of NGINX into their default package lists so maybe you should consider install it from sources.
There is no real difference in security just by changing the process owner name. On the other hand in security, the principle of least privilege states that an entity should be given no more permission than necessary to accomplish its goals within a given system. This way only master process runs as root.
# Edit nginx.conf:
user nginx;
# Set owner and group for root (app, default) directory:
chown -R nginx:nginx /var/www/domain.com
It is recommended to disable any modules which are not required as this will minimize the risk of any potential attacks by limiting the operations allowed by the web server.
The best way to disable unused modules you should use the
configure
option during installation.
# During installation:
./configure --without-http_autoindex_module
# Comment modules in the configuration file e.g. modules.conf:
# load_module /usr/share/nginx/modules/ndk_http_module.so;
# load_module /usr/share/nginx/modules/ngx_http_auth_pam_module.so;
# load_module /usr/share/nginx/modules/ngx_http_cache_purge_module.so;
# load_module /usr/share/nginx/modules/ngx_http_dav_ext_module.so;
load_module /usr/share/nginx/modules/ngx_http_echo_module.so;
# load_module /usr/share/nginx/modules/ngx_http_fancyindex_module.so;
load_module /usr/share/nginx/modules/ngx_http_geoip_module.so;
load_module /usr/share/nginx/modules/ngx_http_headers_more_filter_module.so;
# load_module /usr/share/nginx/modules/ngx_http_image_filter_module.so;
# load_module /usr/share/nginx/modules/ngx_http_lua_module.so;
load_module /usr/share/nginx/modules/ngx_http_perl_module.so;
# load_module /usr/share/nginx/modules/ngx_mail_module.so;
# load_module /usr/share/nginx/modules/ngx_nchan_module.so;
# load_module /usr/share/nginx/modules/ngx_stream_module.so;
Hidden directories and files should never be web accessible - sometimes critical data are published during application deploy. If you use control version system you should defninitely drop the access to the critical hidden directories like a
.git
or.svn
to prevent expose source code of your application.
Sensitive resources contains items that abusers can use to fully recreate the source code used by the site and look for bugs, vulnerabilities, and exposed passwords.
if ($request_uri ~ "../../\.git") {
return 403;
}
# or
location ~ /\.git {
deny all;
}
# or
location ~* ^.*(\.(?:git|svn|htaccess))$ {
return 403;
}
# or all . directories/files excepted .well-known
location ~ /\.(?!well-known\/) {
deny all;
}
Disclosing the version of NGINX running can be undesirable, particularly in environments sensitive to information disclosure.
But the "Official Apache Documentation (Apache Core Features)" say:
Setting ServerTokens to less than minimal is not recommended because it makes it more difficult to debug interoperational problems. Also note that disabling the Server: header does nothing at all to make your server more secure. The idea of "security through obscurity" is a myth and leads to a false sense of safety.
server_tokens off;
In my opinion there is no real reason or need to show this much information about your server. It is easy to look up particular vulnerabilities once you know the version number.
You should compile NGINX from sources with
ngx_headers_more
to usedmore_set_headers
directive.
more_set_headers "Server: Unknown";
When NGINX is used to proxy requests to an upstream server (such as a PHP-FPM instance), it can be beneficial to hide certain headers sent in the upstream response (e.g. the version of PHP running).
proxy_hide_header X-Powered-By;
proxy_hide_header X-AspNetMvc-Version;
proxy_hide_header X-AspNet-Version;
proxy_hide_header X-Drupal-Cache;
Before start see Release Strategy Policies and Changelog on the OpenSSL website.
Criteria for choosing OpenSSL version can vary and it depends all on your use.
The latest versions of the major OpenSSL library are (may be changed):
- the next version of OpenSSL will be 3.0.0
- version 1.1.1 will be supported until 2023-09-11 (LTS)
- last minor version: 1.1.1b (February 26, 2019)
- version 1.1.0 will be supported until 2019-09-11
- last minor version: 1.1.0j (November 20, 2018)
- version 1.0.2 will be supported until 2019-12-31 (LTS)
- last minor version: 1.0.2r (February 26, 2019)
- any other versions are no longer supported
In my opinion the only safe way is based on the up-to-date and still supported version of the OpenSSL. And what's more, I recommend to hang on to the latest versions (e.g. 1.1.1).
If your system repositories do not have the newest OpenSSL, you can do the compilation process (see OpenSSL sub-section).
Advisories recommend 2048 for now. Security experts are projecting that 2048 bits will be sufficient for commercial use until around the year 2030 (as per NIST).
The latest version of FIPS-186 also say the U.S. Federal Government generate (and use) digital signatures with 1024, 2048, or 3072 bit key lengths.
Generally there is no compelling reason to choose 4096 bit keys over 2048 provided you use sane expiration intervals.
If you want to get A+ with 100%s on SSL Lab (for Key Exchange) you should definitely use 4096 bit private keys. That's the main reason why you should use them.
Longer keys take more time to generate and require more CPU (please use
openssl speed rsa
on your server) and power when used for encrypting and decrypting, also the SSL handshake at the start of each connection will be slower. It also has a small impact on the client side (e.g. browsers).
Use of alternative solution: ECC Certificate Signing Request (CSR) -
ECDSA
certificates contain anECC
public key.ECC
keys are better thanRSA & DSA
keys in that theECC
algorithm is harder to break.
The "SSL/TLS Deployment Best Practices" book say:
The cryptographic handshake, which is used to establish secure connections, is an operation whose cost is highly influenced by private key size. Using a key that is too short is insecure, but using a key that is too long will result in "too much" security and slow operation. For most web sites, using RSA keys stronger than 2048 bits and ECDSA keys stronger than 256 bits is a waste of CPU power and might impair user experience. Similarly, there is little benefit to increasing the strength of the ephemeral key exchange beyond 2048 bits for DHE and 256 bits for ECDHE.
Konstantin Ryabitsev (Reddit):
Generally speaking, if we ever find ourselves in a world where 2048-bit keys are no longer good enough, it won't be because of improvements in brute-force capabilities of current computers, but because RSA will be made obsolete as a technology due to revolutionary computing advances. If that ever happens, 3072 or 4096 bits won't make much of a difference anyway. This is why anything above 2048 bits is generally regarded as a sort of feel-good hedging theatre.
My recommendation:
Use 2048-bit key instead 4096-bit at this moment.
### Example (RSA):
( _fd="domain.com.key" ; _len="2048" ; openssl genrsa -out ${_fd} ${_len} )
# Let's Encrypt:
certbot certonly -d domain.com -d www.domain.com --rsa-key-size 2048
### Example (ECC):
# _curve: prime256v1, secp521r1, secp384r1
( _fd="domain.com.key" ; _fd_csr="domain.com.csr" ; _curve="prime256v1" ; \
openssl ecparam -out ${_fd} -name ${_curve} -genkey ; \
openssl req -new -key ${_fd} -out ${_fd_csr} -sha256 )
# Let's Encrypt (from above):
certbot --csr ${_fd_csr} -[other-args]
For x25519
:
( _fd="private.key" ; _curve="x25519" ; \
openssl genpkey -algorithm ${_curve} -out ${_fd} )
( _fd="domain.com.key" ; _len="2048" ; openssl genrsa -out ${_fd} ${_len} )
# Let's Encrypt:
certbot certonly -d domain.com -d www.domain.com
It is recommended to run TLS 1.1/1.2/1.3 and fully disable SSLv2, SSLv3 and TLS 1.0 that have protocol weaknesses.
TLS 1.1 and 1.2 are both without security issues - but only TLS 1.2 and TLS 1.3 provides modern cryptographic algorithms. TLS 1.3 is a new TLS version that will power a faster and more secure web for the next few years. TLS 1.0 and TLS 1.1 protocols will be removed from browsers at the beginning of 2020.
TLS 1.2 does require careful configuration to ensure obsolete cipher suites with identified vulnerabilities are not used in conjunction with it. TLS 1.3 removes the need to make these decisions. TLS 1.3 version also improves TLS 1.2 security, privace and performance issues.
Before enabling specific protocol version, you should check which ciphers are supported by the protocol. So if you turn on TLS 1.1, TLS 1.2 and TLS 1.3 both remember about the correct (and strong) ciphers to handle them. Otherwise, they will not be anyway works without supported ciphers (no TLS handshake will succeed).
If you told NGINX to use TLS 1.3, it will use TLS 1.3 only where is available. NGINX supports TLS 1.3 since version 1.13.0 (released in April 2017), when built against OpenSSL 1.1.1 or more.
My recommendation:
Use only TLSv1.3 and TLSv1.2.
TLS 1.3 + 1.2:
ssl_protocols TLSv1.3 TLSv1.2;
TLS 1.2:
ssl_protocols TLSv1.2;
TLS 1.3 + 1.2 + 1.1:
ssl_protocols TLSv1.3 TLSv1.2 TLSv1.1;
TLS 1.2 + 1.1:
ssl_protocols TLSv1.2 TLSv1.1;
This parameter changes quite often, the recommended configuration for today may be out of date tomorrow.
To check ciphers supported by OpenSSL on your server:
openssl ciphers -s -v
,openssl ciphers -s -v ECDHE
oropenssl ciphers -s -v DHE
.
For more security use only strong and not vulnerable cipher suites. Place
ECDHE
andDHE
suites at the top of your list. The order is important; becauseECDHE
suites are faster, you want to use them whenever clients supports them.
For backward compatibility software components you should use less restrictive ciphers. Not only that you have to enable at least one special
AES128
cipher for HTTP/2 support regarding to RFC7540: TLS 1.2 Cipher Suites, you also have to allowprime256
elliptic curves which reduces the score for key exchange by another 10% even if a secure server preferred order is set.
Also modern cipher suites (e.g. from Mozilla recommendations) suffers from compatibility troubles mainly because drops
SHA-1
. But be careful if you want to use ciphers withHMAC-SHA-1
- there's a perfectly good explanation why.
If you want to get A+ with 100%s on SSL Lab (for Cipher Strength) you should definitely disable
128-bit
ciphers. That's the main reason why you should not use them.
In my opinion
128-bit
symmetric encryption doesn’t less secure. For example TLS 1.3 useTLS_AES_128_GCM_SHA256 (0x1301)
(for TLS-compliant applications). It is not possible to control ciphers for TLS 1.3 without support from client to use new API for TLSv1.3 cipher suites so at this moment it's always on (also if you disable potentially weak cipher from NGINX). On the other hand the ciphers in TLSv1.3 have been restricted to only a handful of completely secure ciphers by leading crypto experts.
For TLS 1.2 you should consider disable weak ciphers without forward secrecy like ciphers with
CBC
algorithm. Using them also reduces the final grade because they don't use ephemeral keys. In my opinion you should use ciphers withAEAD
(TLS 1.3 supports only these suites) encryption because they don't have any known weaknesses.
You should also absolutely disable weak ciphers regardless of the TLS version do you use, like those with
DSS
,DSA
,DES/3DES
,RC4
,MD5
,SHA1
,null
, anon in the name.
We have a nice online tool for testing compatibility cipher suites with user agents: CryptCheck. I think it will be very helpful for you.
My recommendation:
Use only TLSv1.3 and TLSv1.2 with below cipher suites:
ssl_ciphers "TLS13-CHACHA20-POLY1305-SHA256:TLS13-AES-256-GCM-SHA384:TLS13-AES-128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256";
Cipher suites for TLS 1.3:
ssl_ciphers "TLS13-CHACHA20-POLY1305-SHA256:TLS13-AES-256-GCM-SHA384";
Cipher suites for TLS 1.2:
ssl_ciphers "ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES256-SHA384";
Cipher suites for TLS 1.3:
ssl_ciphers "TLS13-CHACHA20-POLY1305-SHA256:TLS13-AES-256-GCM-SHA384:TLS13-AES-128-GCM-SHA256";
Cipher suites for TLS 1.2:
# 1)
ssl_ciphers "ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES256-SHA384";
# 2)
ssl_ciphers "ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256";
# 3)
ssl_ciphers "ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256";
# 4)
ssl_ciphers "EECDH+CHACHA20:EDH+AESGCM:AES256+EECDH:AES256+EDH";
Cipher suites for TLS 1.1 + 1.2:
# 1)
ssl_ciphers "ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256";
# 2)
ssl_ciphers "ECDHE-ECDSA-CHACHA20-POLY1305:ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:!AES256-GCM-SHA256:!AES256-GCM-SHA128:!aNULL:!MD5";
This will also give a baseline for comparison with Mozilla SSL Configuration Generator:
ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';
ssl_ciphers 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS';
For a SSL server certificate, an "elliptic curve" certificate will be used only with digital signatures (
ECDSA
algorithm).
x25519
is a more secure but slightly less compatible option. To maximise interoperability with existing browsers and servers, stick toP-256 prime256v1
andP-384 secp384r1
curves.
NSA Suite B says that NSA uses curves
P-256
andP-384
(in OpenSSL, they are designated as, respectively,prime256v1
andsecp384r1
). There is nothing wrong withP-521
, except that it is, in practice, useless. Arguably,P-384
is also useless, because the more efficientP-256
curve already provides security that cannot be broken through accumulation of computing power.
Use
P-256
to minimize trouble. If you feel that your manhood is threatened by using a 256-bit curve where a 384-bit curve is available, then useP-384
: it will increases your computational and network costs.
If you use TLS 1.3 you should enable
prime256v1
signature algorithm. Without this SSL Lab reportsTLS_AES_128_GCM_SHA256 (0x1301)
signature as weak.
If you do not set
ssh_ecdh_curve
, then NGINX will use its default settings, e.g. Chrome will preferx25519
, but this is not recommended because you can not control default settings (seems to beP-256
) from the NGINX.
Explicitly set
ssh_ecdh_curve X25519:prime256v1:secp521r1:secp384r1;
decreases the Key Exchange SSL Labs rating.
Definitely do not use the
secp112r1
,secp112r2
,secp128r1
,secp128r2
,secp160k1
,secp160r1
,secp160r2
,secp192k1
curves. They have a too small size for security application according to NIST recommendation.
My recommendation:
Use only TLSv1.3 and TLSv1.2 and only strong ciphers with above curves:
ssl_ecdh_curve X25519:secp521r1:secp384r1:prime256v1;
Curves for TLS 1.2:
ssl_ecdh_curve secp521r1:secp384r1:prime256v1;
# Alternative (this one doesn’t affect compatibility, by the way; it’s just a question of the preferred order).
# This setup downgrade Key Exchange score but is recommended for TLS 1.2 + 1.3:
ssl_ecdh_curve X25519:secp521r1:secp384r1:prime256v1;
The DH key is only used if DH ciphers are used. Modern clients prefer
ECDHE
instead and if your NGINX accepts this preference then the handshake will not use the DH param at all since it will not do aDHE
key exchange but anECDHE
key exchange.
Most of the modern profiles from places like Mozilla's ssl config generator no longer recommend using this.
Default key size in OpenSSL is
1024 bits
- it's vulnerable and breakable. For the best security configuration use your own4096 bit
DH Group or use known safe ones pre-defined DH groups (it's recommended) from mozilla.
# To generate a DH key:
openssl dhparam -out /etc/nginx/ssl/dhparam_4096.pem 4096
# To produce "DSA-like" DH parameters:
openssl dhparam -dsaparam -out /etc/nginx/ssl/dhparam_4096.pem 4096
# To generate a ECDH key:
openssl ecparam -out /etc/nginx/ssl/ecparam.pem -name prime256v1
# NGINX configuration:
ssl_dhparam /etc/nginx/ssl/dhparams_4096.pem;
Generally the BEAST attack relies on a weakness in the way CBC mode is used in SSL/TLS.
More specifically, to successfully perform the BEAST attack, there are some conditions which needs to be met:
- vulnerable version of SSL must be used using a block cipher (CBC in particular)
- JavaScript or a Java applet injection - should be in the same origin of the web site
- data sniffing of the network connection must be possible
To prevent possible use BEAST attacks you should enable server-side protection, which causes the server ciphers should be preferred over the client ciphers, and completely excluded TLS 1.0 from your protocol stack.
ssl_prefer_server_ciphers on;
Disable HTTP compression or compress only zero sensitive content.
You should probably never use TLS compression. Some user agents (at least Chrome) will disable it anyways. Disabling SSL/TLS compression stops the attack very effectively. A deployment of HTTP/2 over TLS 1.2 must disable TLS compression (please see RFC 7540: 9.2. Use of TLS Features).
CRIME exploits SSL/TLS compression which is disabled since nginx 1.3.2. BREACH exploits HTTP compression
Some attacks are possible (e.g. the real BREACH attack is a complicated) because of gzip (HTTP compression not TLS compression) being enabled on SSL requests. In most cases, the best action is to simply disable gzip for SSL.
Compression is not the only requirement for the attack to be done so using it does not mean that the attack will succeed. Generally you should consider whether having an accidental performance drop on HTTPS sites is better than HTTPS sites being accidentally vulnerable.
You shouldn't use HTTP compression on private responses when using TLS.
I would gonna to prioritize security over performance but compression can be (I think) okay to HTTP compress publicly available static content like css or js and HTML content with zero sensitive info (like an "About Us" page).
Remember: by default, NGINX doesn't compress image files using its per-request gzip module.
Gzip static module is better, for 2 reasons:
- you don't have to gzip for each request
- you can use a higher gzip level
You should put the
gzip_static on;
inside the blocks that configure static files, but if you’re only running one site, it’s safe to just put it in the http block.
# Disable dynamic HTTP compression:
gzip off;
# Enable dynamic HTTP compression for specific location context:
location / {
gzip on;
...
}
# Enable static gzip compression:
location ^~ /assets/ {
gzip_static on;
...
}
The header indicates for how long a browser should unconditionally refuse to take part in unsecured HTTP connection for a specific domain.
add_header Strict-Transport-Security "max-age=63072000; includeSubdomains" always;
CSP reduce the risk and impact of XSS attacks in modern browsers.
Whitelisting known-good resource origins, refusing to execute potentially dangerous inline scripts, and banning the use of eval are all effective mechanisms for mitigating cross-site scripting attacks.
CSP is a good defence-in-depth measure to make exploitation of an accidental lapse in that less likely.
Before enable this header you should discuss with developers about it. They probably going to have to update your application to remove any inline script and style, and make some additional modifications there.
# This policy allows images, scripts, AJAX, and CSS from the same origin, and does not allow any other resources to load.
add_header Content-Security-Policy "default-src 'none'; script-src 'self'; connect-src 'self'; img-src 'self'; style-src 'self';" always;
Determine what information is sent along with the requests.
add_header Referrer-Policy "no-referrer";
Helps to protect your visitors against clickjacking attacks. It is recommended that you use the
x-frame-options
header on pages which should not be allowed to render a page in a frame.
add_header X-Frame-Options "SAMEORIGIN" always;
Enable the cross-site scripting (XSS) filter built into modern web browsers.
add_header X-XSS-Protection "1; mode=block" always;
It prevents the browser from doing MIME-type sniffing (prevents "mime" based attacks).
add_header X-Content-Type-Options "nosniff" always;
This header protects your site from third parties using APIs that have security and privacy implications, and also from your own team adding outdated APIs or poorly optimized images.
add_header Feature-Policy "geolocation 'none'; midi 'none'; notifications 'none'; push 'none'; sync-xhr 'none'; microphone 'none'; camera 'none'; magnetometer 'none'; gyroscope 'none'; speaker 'none'; vibrate 'none'; fullscreen 'none'; payment 'none'; usb 'none';";
Set of methods support by a resource. An ordinary web server supports the
HEAD
,GET
andPOST
methods to retrieve static and dynamic content. Other (e.g.OPTIONS
,TRACE
) methods should not be supported on public web servers, as they increase the attack surface.
add_header Allow "GET, POST, HEAD" always;
if ($request_method !~ ^(GET|POST|HEAD)$) {
return 405;
}
Buffer overflow attacks are made possible by writing data to a buffer and exceeding that buffers’ boundary and overwriting memory fragments of a process. To prevent this in NGINX we can set buffer size limitations for all clients.
client_body_buffer_size 100k;
client_header_buffer_size 1k;
client_max_body_size 100k;
large_client_header_buffers 2 1k;
Close connections that are writing data too infrequently, which can represent an attempt to keep connections open as long as possible.
client_body_timeout 10s;
client_header_timeout 10s;
keepalive_timeout 5s 5s;
send_timeout 10s;
One of the frequent uses of NGINX is setting it up as a proxy server.
To be completed.
Load balancing is a useful mechanism to distribute incoming traffic around several capable servers. We may improve of some rules about the NGINX working as a load balancer.
Monitoring for health is important on all types of load balancing mainly for business continuity. Passive checks watches for failed or timed-out connections as they pass through NGINX as requested by a client.
This functionality is enabled by default but the parameters mentioned here allow you to tweak their behavior. Default values are:
max_fails=1
andfail_timeout=10s
.
upstream backend {
server bk01_node:80 max_fails=3 fail_timeout=5s;
server bk02_node:80 max_fails=3 fail_timeout=5s;
}
down
parameterSometimes we need to turn off backends e.g. at maintenance-time. I think good solution is marks the server as permanently unavailable with
down
parameter even if the downtime takes a short time.
Comments are good for really permanently disable servers or if you want to leave information for historical purposes.
NGINX also provides a
backup
parameter which marks the server as a backup server. It will be passed requests when the primary servers are unavailable. I use this option rarely for the above purposes and only if I am sure that the backends will work at the maintenance time.
upstream backend {
server bk01_node:80 max_fails=3 fail_timeout=5s down;
server bk02_node:80 max_fails=3 fail_timeout=5s;
}
This rules aren't strictly related to the NGINX but in my opinion they're also very important aspect of security.
DNS CAA policy helps you to control which Certificat Authorities are allowed to issue certificates for your domain becaues if no CAA record is present, any CA is allowed to issue a certificate for the domain.
Generic configuration (Google Cloud DNS, Route 53, OVH, and other hosted services) for Let's Encrypt:
example.com. CAA 0 issue "letsencrypt.org"
Standard Zone File (BIND, PowerDNS and Knot DNS) for Let's Encrypt:
example.com. IN CAA 0 issue "letsencrypt.org"
security.txt
The main purpose of
security.txt
is to help make things easier for companies and security researchers when trying to secure platforms. It also provides information to assist in disclosing security vulnerabilities.
When security researchers detect potential vulnerabilities in a page or application, they will try to contact someone "appropriate" to "responsibly" reveal the problem. It's worth taking care of getting to the right address.
This file should be placed under the
/.well-known/
path, e.g./.well-known/security.txt
(RFC5785) of a domain name or IP address for web properties.
curl -ks https://example.com/.well-known/security.txt
Contact: security@example.com
Contact: +1-209-123-0123
Encryption: https://example.com/pgp.txt
Preferred-Languages: en
Canonical: https://example.com/.well-known/security.txt
Policy: https://example.com/security-policy.html
And from Google:
curl -ks https://www.google.com/.well-known/security.txt
Contact: https://g.co/vulnz
Contact: mailto:security@google.com
Encryption: https://services.google.com/corporate/publickey.txt
Acknowledgements: https://bughunter.withgoogle.com/
Policy: https://g.co/vrp
Hiring: https://g.co/SecurityPrivacyEngJobs
# Flag: BountyCon{075e1e5eef2bc8d49bfe4a27cd17f0bf4b2b85cf}
Remember to make a copy of the current configuration and all files/directories.
I used step-by-step tutorial from Installation from source.
Configuration of Google Cloud instance:
ITEM | VALUE | COMMENT |
---|---|---|
VM | Google Cloud Platform | |
vCPU | 2x | |
Memory | 4096MB | |
HTTP | Varnish on port 80 | |
HTTPS | NGINX on port 443 |
This chapter describes the basic configuration of my proxy server (for blkcipher.info domain).
Configuration of my Reverse Proxy server is based on installation from source chapter. If you go through the installation process step by step you can use the following configuration (minor adjustments may be required).
It's very simple - clone the repo, backup your current configuration and perform full directory sync:
git clone https://github.com/trimstray/nginx-admins-handbook
tar czvfp ~/nginx.etc.tgz /etc/nginx && mv /etc/nginx /etc/nginx.old
rsync -avur lib/nginx/ /etc/nginx/
If you compiled NGINX you should also update/refresh modules. All compiled modules are stored in
/usr/local/src/nginx-${ngx_version}/master/objs
and installed in accordance with the value of the--modules-path
variable.
cd /etc/nginx
find . -not -path '*/\.git*' -depth -name '*192.168.252.2*' -execdir bash -c 'mv -v "$1" "${1//192.168.252.2/xxx.xxx.xxx.xxx}"' _ {} \;
cd /etc/nginx
find . -not -path '*/\.git*' -type f -print0 | xargs -0 sed -i 's/192.168.252.2/xxx.xxx.xxx.xxx/g'
cd /etc/nginx
find . -not -path '*/\.git*' -depth -name '*blkcipher.info*' -execdir bash -c 'mv -v "$1" "${1//blkcipher.info/example.com}"' _ {} \;
cd /etc/nginx
find . -not -path '*/\.git*' -type f -print0 | xargs -0 sed -i 's/blkcipher_info/example_com/g'
find . -not -path '*/\.git*' -type f -print0 | xargs -0 sed -i 's/blkcipher.info/example.com/g'
cd /etc/nginx/master/_server/localhost/certs
# Private key + Self-signed certificate:
( _fd="localhost.key" ; _fd_crt="nginx_localhost_bundle.crt" ; \
openssl req -x509 -newkey rsa:2048 -keyout ${_fd} -out ${_fd_crt} -days 365 -nodes \
-subj "../../C=X0/ST=localhost/L=localhost/O=localhost/OU=X00/CN=localhost" )
default_server
cd /etc/nginx/master/_server/defaults/certs
# Private key + Self-signed certificate:
( _fd="defaults.key" ; _fd_crt="nginx_defaults_bundle.crt" ; \
openssl req -x509 -newkey rsa:2048 -keyout ${_fd} -out ${_fd_crt} -days 365 -nodes \
-subj "../../C=X1/ST=default/L=default/O=default/OU=X11/CN=default_server" )
cd /etc/nginx/master/_server/example.com/certs
# For multidomain:
certbot certonly -d example.com -d www.example.com --rsa-key-size 2048
# For wildcard:
certbot certonly --manual --preferred-challenges=dns -d example.com -d *.example.com --rsa-key-size 2048
# Copy private key and chain:
cp /etc/letsencrypt/live/example.com/fullchain.pem nginx_example.com_bundle.crt
cp /etc/letsencrypt/live/example.com/privkey.pem example.com.key
Update modules list and include modules.conf
to your configuration:
_mod_dir="../../etc/nginx/modules"
:>"${_mod_dir}.conf"
for _module in $(ls "${_mod_dir}/") ; do echo -en "load_module\t\t${_mod_dir}/$_module;\n" >> "${_mod_dir}.conf" ; done
In the example (
lib/nginx
) error pages are included fromlib/nginx/master/_static/errors.conf
file.
/etc/nginx/html
: 50x.html index.html
/usr/share/www
: cd /etc/nginx/snippets/http-error-pages
./httpgen
# You can also sync sites/ directory with /etc/nginx/html:
# rsync -var sites/ /etc/nginx/html/
rsync -var sites/ /usr/share/www/
nginx.conf
# At the end of the file (in 'IPS/DOMAINS' section):
include /etc/nginx/master/_server/domain.com/servers.conf;
include /etc/nginx/master/_server/domain.com/backends.conf;
cd /etc/nginx/cd master/_server
cp -R example.com domain.com
cd domain.com
find . -not -path '*/\.git*' -depth -name '*example.com*' -execdir bash -c 'mv -v "$1" "${1//example.com/domain.com}"' _ {} \;
find . -not -path '*/\.git*' -type f -print0 | xargs -0 sed -i 's/example_com/domain_com/g'
find . -not -path '*/\.git*' -type f -print0 | xargs -0 sed -i 's/example.com/domain.com/g'
mkdir -p /var/log/nginx/localhost
mkdir -p /var/log/nginx/defaults
mkdir -p /var/log/nginx/others
mkdir -p /var/log/nginx/domains/blkcipher.info
chown -R nginx:nginx /var/log/nginx
cp /etc/nginx/snippets/logrotate.d/nginx /etc/logrotate.d/
nginx -t -c /etc/nginx/nginx.conf