Forum

Hiawatha version: 9.6, the latest
Operating System: debian 7 32bit

Hiawatha is good Secure server (WAF) ,simple and easy to use . You can install it , needn't think about more how to configure secure setting.

But these days i find that hiawatha reverseproxy is slower than nginx reverseproxy. The firebug shows my website is opened in 5s with nginx, and 1m 5s with hiawatha.

That is so slow . And I added "CacheRProxyExtensions = css,gif,html,jpg,js,png,txt" , hiawatha become a liitle bit faster now. hiawatha is slow in js which shows in firebug.

You can test the speed of my website:

hiawatha: http://www.bhzhu203.com:90/

nginx: http://www.bhzhu203.com/

Here is the config of my hiawatha :

CacheRProxyExtensions = css,gif,html,jpg,js,png,txt

VirtualHost {
    Hostname = bbs.bhzhu203.com  xxx.com
WebsiteRoot = /var/www/hiawatha
    ReverseProxy .* http://127.0.0.1:8080/  125
#AccessLogfile = /home/log/hiawatha/main-access.log
ErrorLogfile = /var/log/hiawatha/main-error.log
}

My website is in 2 severs : nginx(hiawatha) < varnish < hiawatha < nginx

Hope to solve this problem. Thanks

By the way , i use a http dos script to test the server network output for hack :

hiwatha : 1.5 Mb/s output average

senginx : 400kb/s output average

So , hiwatha is not good enough in DOS prevention. senginx is for secure too, it has graet speed and WAF fuction ,you can know about it .


Hope hiwatha to be graeter ,thanks
.

Sorry ! It is my fault , I used hiawatha default page (static html) to test not real web site . So I had got the hight speed.

For normal php page ,hiwatha can nearly ban all DOS actions ,and throw out 40X errors ，performing better than senignx.


The left probrom is the ReverseProxy speed

I'm back from holiday. Can you put the Hiawatha reverse proxy back online? I want to take a look for myself.

http://www.bhzhu203.com:90/ is online now ,please make a test

Ok, will do a test tomorrow morning.

I did some testing on both port 80 and 90, but I saw some issues on both ports. I used telnet and entered the following request:

GET / HTTP/1.0
Host: www.bhzhu203.com

Sometimes I get a full response, sometimes it stops halfway the response. Perhaps the cause is not at the proxy, but at the final webserver. It's hard for me to tell what causes this.

The ReverseProxy sever is location in Japan ,maybe you have network package lost there. You can traceroute to find if there is network package lost .


It is a good choice to use firebug of firefox to test the network response of these two servers : nginx , hiawatha.

these pictures show something:

It's not a case of package loss. It's really a case of a not-responding server.

My guess is that is has something to do with the length of the request or request-chunk not matching the length specified in the request header or chunk header. Maybe due to all the Chinese characters in the content?

I don't know the exact reason of that.

Here are more pictures to show:

1. In firefox web browser , it usally takes a long time to completely open a page in my website for hiawatha



2.usally "Typesetting confusion"






3.the same file, different speed





Now , I have changed the DNS of www.bhzhu203.com to the origin server location in the US.

So, you can continue making the test .

www.bhzhu203.com hiawatha server
www.bhzhu203.com:100 Nginx server

If I request a file without any Chinese characters (the bootstrap CSS file for example), everything goes well. If I request only the HTML page which does contain Chinese characters, it goes wrong. My guess is that the Chinese UTF-8 characters mess up the content length calculation at the web server, which makes the proxy get confused.

Thanks for testing.

Could you solve this issue in the next hiawatha version ? Maybe you could learn from Nginx which has its way to deal with the Chinese UTF-8 characters .

I have to switch to SeNginx now, because i have websites which need for fast open.

If you have any test for this , please mail to me.

I am hoping for next hiawatha version to be faster.

Best wishes

There is not something for me to solve. A webserver should not have to worry about some character encoding of the content. So, the specified length should match the length of the raw content.

Why does “Chinese UTF-8 characters mess up the content length calculation ” happen ？ Is the problem of Nginx？


Nginx is behide Hiawatha . Whose wrong it is ? How to solve it?


Thanks.

First, I'm not sure that the utf-8 characters are the cause of your issue, it's my guess. It's my guess, because incorrect Content-Length or chunk lengths within the request are often the cause in such situation. It's something for you to investigate.

An utf-8 character consists of two characters. So, when you see only one character on your screen, it's actually two bytes that has been sent. The correct Content-Length in that case should be 2, not 1. If nginx uses a length of 1, then nginx is wrong. Read this article [mark.koli.ch] for more information about this.

Now , I have found out why I open my webpage slow with hiawatha.

Use chrome F12 ,network, then open any page of www.bhzhu203.com , you can see the red words below:

(failed)
net::ERR_INCOMPLETE_CHUNKED_ENCODING

Nginx uses gzip compression behind Hiawatha.

php also ues compression.

Is there a way I setup a proxy at my own server, which directly connects to your final website? This way I can see what goes on in Hiawatha internally while proxying your website.

Here is my final server 107.161.26.233 ： nginx （gzip compression）> nginx (none gzip 107.161.26.233 )

The final nginx(gziped) is in the Intranet .

domains: www.bhzhu203.com www.swordm.com www.plkq.net


The test IP is 107.161.26.233

Add "141.138.201.249 www.bhzhu203.com" to the hosts file of your system (/etc/hosts for Linux and Mac or C:\system32\drivers\etc\hosts for Windows). Restart your browser and visit www.bhzhu203.com. It will then go via my webserver. I've tested Hiawatha's reverse proxy via this and it all seems to work fine. What about you?

Yes, it is no problem now , opening fast with hiawatha 9.7.

curl -i http://141.138.201.249
HTTP/1.1 200 OK
Date: Wed, 20 Aug 2014 16:25:34 GMT
Server: Hiawatha v9.7
Connection: keep-alive
X-Empty:                                                       
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Hiawatha-Cache: 300
X-Frame-Options: sameorigin
Content-Type: text/html; charset=utf-8
Content-Language: en
Content-Length: 136
X-Powered-By: Banshee PHP framework v3.4

But my hiawatha has no cache ?
curl -i se.bhzhu203.com
HTTP/1.1 200 OK
Date: Wed, 20 Aug 2014 16:45:10 GMT
Server: Hiawatha v9.6
Connection: keep-alive
Content-Type: text/html
Last-Modified: Sun, 01 Jun 2014 08:03:18 GMT
Content-Length: 1566


What is wrong in my hiawatha.conf?

# Hiawatha main configuration file
#


# GENERAL SETTINGS
#
ServerId = www
ConnectionsTotal = 950
ConnectionsPerIP = 80
SystemLogfile = /home/log/hiawatha/system.log
GarbageLogfile = /home/log/hiawatha/garbage.log


# BINDING SETTINGS
# A binding is where a client can connect to.
#
Binding {
	Port = 80
	Interface = 107.161.xxxxxx
#	MaxKeepAlive = 30
#	TimeForRequest = 3,20
}


Binding {
        Port = 80
        Interface = 107.161.19.121
#       MaxKeepAlive = 30
#       TimeForRequest = 3,20
}






#
#Binding {
#	Port = 443
#	Interface = ::1
#	MaxKeepAlive = 30
#	TimeForRequest = 3,20
#	SSLcertFile = hiawatha.pem
#}


# BANNING SETTINGS
# Deny service to clients who misbehave.
#
#BanOnGarbage = 300
#BanOnMaxPerIP = 60
#BanOnMaxReqSize = 300
#KickOnBan = yes
#RebanDuringBan = yes


# COMMON GATEWAY INTERFACE (CGI) SETTINGS
# These settings can be used to run CGI applications.
#
#CGIhandler = /usr/bin/perl:pl
#CGIhandler = /usr/bin/php-cgi:php
#CGIhandler = /usr/bin/python:py
#CGIhandler = /usr/bin/ruby:rb
#CGIhandler = /usr/bin/ssi-cgi:shtml
#CGIextension = cgi
#
#FastCGIserver {
#	FastCGIid = PHP5
#	ConnectTo = 172.17.0.2:9000
#	Extension = php
#}


# URL TOOLKIT
# This URL toolkit rule was made for the Banshee PHP framework, which
# can be downloaded from http://www.hiawatha-webserver.org/banshee
#
#UrlToolkit {
#	ToolkitID = banshee
#	RequestURI isfile Return
#	Match ^/(css|files|images|js|slimstat)($|/) Return
#	Match ^/(favicon.ico|robots.txt|sitemap.xml)$ Return
#	Match .*\?(.*) Rewrite /index.php?$1
#	Match .* Rewrite /index.php
#}


# DEFAULT WEBSITE
# It is wise to use your IP address as the hostname of the default website
# and give it a blank webpage. By doing so, automated webscanners won't find
# your possible vulnerable website.
#
Hostname = 127.0.0.1
WebsiteRoot = /var/www/hiawatha
StartFile = index.html
AccessLogfile = /home/log/hiawatha/access.log
ErrorLogfile = /home/log/hiawatha/error.log
#ErrorHandler = 404:/error.cgi


# VIRTUAL HOSTS
# Use a VirtualHost section to declare the websites you want to host.
#
#VirtualHost {
#	Hostname = www.my-domain.com
#	WebsiteRoot = /var/www/my-domain/public
#	StartFile = index.php
#	AccessLogfile = /var/www/my-domain/log/access.log
#	ErrorLogfile = /var/www/my-domain/log/error.log
#	TimeForCGI = 5
#	UseFastCGI = PHP5
#	UseToolkit = banshee
#}


# DIRECTORY SETTINGS
# You can specify some settings per directory.
#
#Directory {
#	Path = /home/baduser
#	ExecuteCGI = no
#	UploadSpeed = 10,2
#}


CacheRProxyExtensions = css,gif,html,jpg,js,png,txt

VirtualHost {    
    Hostname = buy.plkq.net,bbs.datangjie.com,bbs.bhzhu203.com
    
WebsiteRoot = /var/www/hiawatha
#    ReverseProxy .* http://172.17.0.7:8080/  95   keep-alive  
 ReverseProxy .* http://127.0.0.1:884/  95   keep-alive
#AccessLogfile = /home/log/hiawatha/main-access.log
ErrorLogfile = /home/log/hiawatha/main-error.log
#ErrorHandler = 504:/504.html
#ErrorHandler = 508:/508.html

#TimeForCGI = 30

}

Why do you think Hiawatha has no cache? Use 'hiawatha -v' to see what modules have been compiled into Hiawatha. But Hiawatha will not cache the response you showed because of the 'no-cache' in it.

hiawatha -v
Hiawatha v9.6, cache, IPv6, reverse proxy, SSL (1.3.7), URL toolkit, XSLT
Copyright (c) by Hugo Leisink <hugo@leisink.net>

It has the cache module.

But your hiawatha 9.7 has this " X-Hiawatha-Cache: 300",mine having not.

Does it has "no-cache" will be fast?

Did the problem be solved in hiawatha 9.7?

The isuee has not been solved in hiawatha 9.7 yet ,opening slowly.

build it whit "-DENABLE_DEBUG=on" ,you can find when opened slowly it shows again and again like this :

Filename:    /home/ubuntu/hiawatha-9.7/src/hiawatha.c
Line number: 404
Memory size: 3368
08 10 31 09 00 00 00 00 70 AF FC 53 34 00 00 00   |..1.....p..S4...|
21 00 00 00 D0 36 31 09 01 01 01 00 06 00 00 00   |!....61.........|
00 00 00 00 00 00 00 00 00 00 00 00 C0 1A 40 B6   |..............@.|
C0 1A 40 B6 C4 1A 40 B6 00 00 00 00 CF 1A 40 B6   |..@...@.......@.|
0F 1B 40 B6 00 00 00 00 70 0E 40 B6 5D 03 00 00   |..@.....p.@.]...|
00 00 00 00 00 10 00 00 5D 03 00 00 01 00 00 00   |........].......|
00 00 00 00 30 31 40 B6 4A 00 00 00 00 00 00 00   |....01@.J.......|
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   |................|
01 00 00 00 68 2B 40 B6 02 00 00 00 7D 74 08 D6   |....h+@.....}t..|
00 00 00 00 00 00 00 00 00 00 00 00 04 00 00 00   |................|

Filename:    /home/ubuntu/hiawatha-9.7/src/hiawatha.c
Line number: 962
Memory size: 32
03 00 00 00 13 00 00 00 5C 53 2B 29 3F 49 4E 5C   |........\S+)?IN\|
53 2A 5C 28 5C 53 2A 5B 27 30 2D 39 5D 00 00 00   |S*\(\S*['0-9]...|

Total memory usage: 167686

--[ T ]--------------------------
--[ T ]--------------------------
--[ T ]--------------------------
--[ T ]--------------------------

What is the content of your hiawatha conf to proxy my website ? Please show to me ,thanks

How to set "Content-Type: text/html; charset=utf-8" in hiawatha?

VirtualHost {
    Hostname = www.bhzhu203.com
    WebsiteRoot = /var/www/bhzhu203
    AccessLogfile = /var/log/hiawatha/bhzhu203-access.log
    ErrorLogfile = /var/log/hiawatha/bhzhu203-error.log
    ReverseProxy .* http://107.161.26.233/
}

About the 'charset=utf-8', it should be printed by a CGI or you can use:

<meta http-equiv="Content-Type" content="text/html; charset=utf-8">

Great ! Thanks Hugo,

When I use " ReverseProxy .* http://xx.xx.xx.xx " without "keep-alive" in Hiawatha 9.7, my website can be browsed with fast speed , 1.5s per page average , even faster than nginx！

It is the problem in ReverseProxy with ”keep-alive“ option.


Hiawatha is an excellent secure webserver ,I will introduce it to more people.

Please, try the following. Edit src/target.c. In find_chunk_size() change

    if ((*chunk_size = hex_to_int(buffer)) == -1) {

to

    *c = '\0';
    *chunk_size = hex_to_int(buffer);
    *c = '\r';

    if (*chunk_size == -1) {

Enable the keep-alive option again for the reverse proxy. Please, let me know if this solves the issue.

It is OK now,when I enable the keep-alive option again for the reverse proxy,opening fast really.

I think this issue has been solved already. Well done !


I will continue watching it.