-
-
Save hrchu/0cc5e783387f5453f528 to your computer and use it in GitHub Desktop.
| # Follows the squid format in default: | |
| # logformat squid %ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %ru %[un %Sh/%<a %mt | |
| # http://www.squid-cache.org/Doc/config/logformat/ | |
| %{NUMBER:timestamp}\s+%{NUMBER:response_time} %{IPORHOST:src_ip} %{WORD:squid_request_status}/%{NUMBER:http_status_code} %{NUMBER:reply_size_include_header} %{WORD:http_method} %{URI:request_url} %{USERNAME:user} %{WORD:squid_hierarchy_status}/%{IPORHOST:server_ip_or_peer_name} (?<mime_content_type>\S+\/\S+) |
%{POSINT:timestamp}.%{POSINT:timestamp_ms}\s+%{NUMBER:response_time} %{IPORHOST:src_ip} %{WORD:squid_request_status}/%{NUMBER:http_status_code} %{NUMBER:reply_size_include_header} %{WORD:http_method} %{NOTSPACE:request_url} %{NOTSPACE:user} %{WORD:squid}/%{IP:server_ip} %{NOTSPACE:content_type}
I still get some failures on HTTPS urls with your pattern ebuildy. One example is:
1446816499.069 648 1.1.1.1 TCP_MISS/200 2415 CONNECT ent-shasta-rrs.symantec.com:443 - FIRSTUP_PARENT/185.46.212.34 -
%{POSINT:timestamp}.%{POSINT:timestamp_ms}\s+%{NUMBER:response_time}\s+%{IPORHOST:user} %{WORD:result}/%{NUMBER:status_codes} %{NUMBER:transfer_size} %{WORD:request_method} (?=%{NOTSPACE:request_url})((?:(http://)%{IPORHOST:domain}(?::%{POSINT:port})?)(?:|%{URIPATH:url_path})(?:|%{URIPARAM:url_querystring})) (?:-|%{NOTSPACE:client_identity}) %{WORD:peering_code}/(?:-|%{NOTSPACE:peerhost}) (?:-|%{NOTSPACE:content_type})
This one works even if squid is sending access logs to a remote syslog server.
(%{DATA:head}:)?\s+%{NUMBER:timestamp}\s+%{NUMBER:response_time} %{IPORHOST:src_ip} %{NOTSPACE:squid_request_status}/%{NUMBER:http_status_code} %{NUMBER:transfer_size} %{NOTSPACE:http_method} (%{URIPROTO:proto}://)?%{NOTSPACE:request_url}(:%{INT:tcp_port})?(/%{NOTSPACE:url_path})?\s+%{NOTSPACE:client_identity}\s+%{NOTSPACE:peer_code}/%{NOTSPACE:peerhost}\s+%{NOTSPACE:content_type}
Here's one that properly breaks the request URI into scheme, host, port, and path. NOTSPACE can't be used for the host since it's greedy and will prevent the port and path from matching.
%{NUMBER:timestamp}\s+%{NUMBER:response_time} %{IPORHOST:src_ip} %{NOTSPACE:squid_request_status}/%{NUMBER:http_status_code} %{NUMBER:transfer_size} %{NOTSPACE:http_method} (%{URIPROTO:url_scheme}://)?(?<url_host>\S+?)(:%{INT:url_port})?(/%{NOTSPACE:url_path})?\s+%{NOTSPACE:client_identity}\s+%{NOTSPACE:peer_code}/%{NOTSPACE:peerhost}\s+%{NOTSPACE:content_type}
Dear friends,
In my environment I have squid on pfsense 2.4.3 and I have problems to see application and url on squid logs in my logstash. Can you help me with this configuration? Anyone know how to send this complete log to logstash, can you send me the step by step? that's my sample log message. I don´t see the apllication
message:<166>Sep 18 08:57:17 (squid-1): 1537271837.019 119300 10.32.211.43 TCP_TUNNEL/200 424889 CONNECT www.reclameaqui.com.br:443 joaojbm HIER_DIRECT/23.4.77.209 -
This not working for https. (URI should be NOTSPACE):
1438694263.977 7356 172.17.1.161 TCP_MISS/200 0 CONNECT graph.facebook.com:443 - HIER_DIRECT/127.0.0.1 -