I enabled the file commenting out Include conf/extra/httpd-ssl.conf in /etc/httpd/conf/httpd.conf
I had errors about DEFLATE so I commented it.
AH00526: Syntax error on line 16 of /etc/httpd/conf/extra/httpd-ssl.conf:
Unknown filter provider DEFLATE
Then I commented also SSLCertificateFile and SSLCertificateFile because I do not have them yet.
Apache won't start and I do not see anymore errors in the log.
This is how I cloned the repo.
cd /srv/http
git clone https://git.lolcat.ca/lolcat/4get
mv 4get c
cd c
chmod 777 -R icons/
I am not sure if I specified DocumentRoot /srv/http/c, RedirectMatch 301 ^(.*)$ https://c.ca$1 etc. correctly.
Is it fine asking here?
I am looking the guide https://git.lolcat.ca/lolcat/4get/src/branch/master/docs/apache2.md.
Artix instructions should be the same for Arch (except the service commands).
I did the step before https://git.lolcat.ca/lolcat/4get/src/branch/master/docs/apache2.md#default-sslconf but I may have missed something.
`/etc/httpd/confextra/httpd-ssl.conf`:
```
<VirtualHost *:443>
RedirectMatch 301 ^(.*)$ https://http://localhost/c.ca/$1
ServerAdmin user@c.ca
#ErrorLog ${APACHE_LOG_DIR}/error.log
SSLEngine on
<FilesMatch "\.(?:cgi|shtml|phtml|php)$">
SSLOptions +StdEnvVars
</FilesMatch>
<Directory /usr/lib/cgi-bin>
SSLOptions +StdEnvVars
</Directory>
#AddOutputFilterByType DEFLATE application/json
#AddOutputFilterByType DEFLATE application/javascript
#AddOutputFilterByType DEFLATE application/x-javascript
#AddOutputFilterByType DEFLATE text/html
#AddOutputFilterByType DEFLATE text/plain
#AddOutputFilterByType DEFLATE text/css
#SSLCertificateFile /etc/letsencrypt/live/4get.ca/fullchain.pem
#SSLCertificateKeyFile /etc/letsencrypt/live/4get.ca/privkey.pem
</VirtualHost>
<VirtualHost *:443>
ServerName c.ca
DocumentRoot /srv/http/c
Options +MultiViews
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^([^\.]+)$ $1.php [NC,L]
# deny access to private resources
<Directory /srv/http/c/>
Order Deny,allow
Deny from all
</Directory>
</VirtualHost>
<VirtualHost *:443>
ServerName c.ca
DocumentRoot /srv/http/c
Options +MultiViews
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^([^\.]+)$ $1.php [NC,L]
</VirtualHost>
<VirtualHost *:443>
ServerName www.c.ca
RedirectMatch 301 ^(.*)$ https://c.ca$1
</VirtualHost>
```
I enabled the file commenting out `Include conf/extra/httpd-ssl.conf` in `/etc/httpd/conf/httpd.conf`
I had errors about `DEFLATE` so I commented it.
```
AH00526: Syntax error on line 16 of /etc/httpd/conf/extra/httpd-ssl.conf:
Unknown filter provider DEFLATE
```
Then I commented also `SSLCertificateFile` and `SSLCertificateFile` because I do not have them yet.
Apache won't start and I do not see anymore errors in the log.
This is how I cloned the repo.
```
cd /srv/http
git clone https://git.lolcat.ca/lolcat/4get
mv 4get c
cd c
chmod 777 -R icons/
```
I am not sure if I specified `DocumentRoot /srv/http/c`, `RedirectMatch 301 ^(.*)$ https://c.ca$1` etc. correctly.
Is it fine asking here?
For the deflate error, please try enabling the extension using sudo a2enmod enable deflate or something like that. Also try with mod-deflate if that doesn't work. If the module doesn't exist, try and look in your package manager for apache2 package extras.
That RedirectMatch directive will redirect all traffic on port 443 (https, assuming SSLEngine is on) to whatever domain you specify. I'm not sure what you're trying to do here, are you trying to only allow traffic through localhost? If so, leave out the ssl config stuff. If not, try and fix it like this, assuming you're using the c.ca domain:
RedirectMatch 301 ^(.*)$ https://c.ca/$1
Again, this will redirect all SSL traffic to your c.ca domain, so make sure that this is what you want.
Now, Apache2 won't start because there is no SSLCertificateFile while SSLEngine is on. Generate your certificates and uncomment the SSLCertificateFile directives, all the while making sure you supply the right file paths.
Also, not quite sure why you have 2 rulesets with ServerName c.ca? It's better to merge them to avoid conflicts.
Keep me posted!
Asking here is fine, lol
For the deflate error, please try enabling the extension using `sudo a2enmod enable deflate` or something like that. Also try with `mod-deflate` if that doesn't work. If the module doesn't exist, try and look in your package manager for apache2 package extras.
That `RedirectMatch` directive will redirect all traffic on port 443 (https, assuming `SSLEngine` is on) to whatever domain you specify. I'm not sure what you're trying to do here, are you trying to only allow traffic through `localhost`? If so, leave out the ssl config stuff. If not, try and fix it like this, assuming you're using the `c.ca` domain:
`RedirectMatch 301 ^(.*)$ https://c.ca/$1`
Again, this will redirect all SSL traffic to your c.ca domain, so make sure that this is what you want.
Now, Apache2 won't start because there is no `SSLCertificateFile` while `SSLEngine` is on. Generate your certificates and uncomment the `SSLCertificateFile` directives, all the while making sure you supply the right file paths.
Also, not quite sure why you have 2 rulesets with `ServerName c.ca`? It's better to merge them to avoid conflicts.
Keep me posted!
I'd like to set up my 4get instance that I can use locally so I supposed that you use https://http://localhost/c.ca in these cases.
I am not sure to how set up the local instance.
I made 2 rulesets trying to following the guide but I am not familiar with this.
In /etc/httpd/conf/httpd.conf I now commented out LoadModule deflate_module modules/mod_deflate.so, looks like that a2enmod is Debian specific.
In this case do I need ssl config?
I'd like to set up my 4get instance that I can use locally so I supposed that you use `https://http://localhost/c.ca` in these cases.
I am not sure to how set up the local instance.
I made 2 rulesets trying to following the guide but I am not familiar with this.
In `/etc/httpd/conf/httpd.conf` I now commented out `LoadModule deflate_module modules/mod_deflate.so`, looks like that `a2enmod` is Debian specific.
In this case do I need ssl config?
You do not need an ssl config for local use. Why do you insist on using https://http://localhost/c.ca ? This is wrong, you specify the protocol 2 times. Since you're running this locally, there is no need to setup redirects, you can just access your webserver directly with http://localhost .
Please let me know if you have any other questions :)
You do not need an ssl config for local use. Why do you insist on using `https://http://localhost/c.ca` ? This is wrong, you specify the protocol 2 times. Since you're running this locally, there is no need to setup redirects, you can just access your webserver directly with http://localhost .
Please let me know if you have any other questions :)
Could you make an example of all the things that I need?
Not sure where put those rules in `/etc/httpd/conf/httpd.conf`?
Or
```
ls /etc/httpd/conf/extra/
httpd-autoindex.conf httpd-info.conf httpd-mpm.conf httpd-userdir.conf
httpd-dav.conf httpd-languages.conf httpd-multilang-errordoc.conf httpd-vhosts.conf
httpd-default.conf httpd-manual.conf httpd-ssl.conf proxy-html.conf
```
At this point, what I would suggest doing would be to run dpkg-reconfigure on your apache2 webserver and create an .htaccess configuration file in your /var/www/4get/ folder that can get rid of the .php extension by doing an internal URL rewrite. This is all you need to do in order to make 4get work locally.
You can look up how to enable .htaccess online and then just place the following rules in there:
When you make configuration changes, make sure to restart apache2 every time.
Please let me know if you encounter any other issues.
At this point, what I would suggest doing would be to run `dpkg-reconfigure` on your apache2 webserver and create an `.htaccess` configuration file in your `/var/www/4get/` folder that can get rid of the `.php` extension by doing an internal URL rewrite. This is all you need to do in order to make 4get work locally.
You can look up how to enable .htaccess online and then just place the following rules in there:
```
Options +MultiViews
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^([^\.]+)$ $1.php [NC,L]
```
When you make configuration changes, make sure to restart apache2 every time.
Please let me know if you encounter any other issues.
/srv/http is the defautl value in the conf, and I enabled AllowOverride:
DocumentRoot "/srv/http"
<Directory "/srv/http">
Options Indexes FollowSymLinks
AllowOverride All
Require all granted
</Directory>
<Directory />
AllowOverride All
Require all denied
</Directory>
When I go to localhost I see the 4get folder and I can browse it but I do not know to how use it as search engine.
For local use do I need all the packages php-mbstring apache2 certbot php-imagick imagemagick php-curl curl php-apcu git libapache2-mod-php?
And ssl, rewrite, proxy and headers modules?
https://httpd.apache.org/docs/current/howto/htaccess.html#when
I made this in the conf and I also made the equivalent `.htaccess` file in `/srv/http/4get`.
```
<Directory "/srv/http/4get">
Options +MultiViews
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^([^\.]+)$ $1.php [NC,L]
</Directory>
```
`/srv/http` is the defautl value in the conf, and I enabled `AllowOverride`:
```
DocumentRoot "/srv/http"
<Directory "/srv/http">
Options Indexes FollowSymLinks
AllowOverride All
Require all granted
</Directory>
<Directory />
AllowOverride All
Require all denied
</Directory>
```
When I go to localhost I see the 4get folder and I can browse it but I do not know to how use it as search engine.
For local use do I need all the packages `php-mbstring apache2 certbot php-imagick imagemagick php-curl curl php-apcu git libapache2-mod-php`?
And ssl, rewrite, proxy and headers modules?
You need php-mbstring apache2 php-imagick imagemagick php-curl curl php-apcu libapache2-mod-php and rewrite modules to get rid of the .php extension.
You need `php-mbstring apache2 php-imagick imagemagick php-curl curl php-apcu libapache2-mod-php` and rewrite modules to get rid of the .php extension.
I don't really understand why you're having this many issues. I have followed these installation steps myself multiple times now while migrating machines and I can assure you that they work.
When you access your webserver, does anything render, or is it just a directory listing? When you access /index.php, what happens?
I don't really understand why you're having this many issues. I have followed these installation steps myself multiple times now while migrating machines and I can assure you that they work.
When you access your webserver, does anything render, or is it just a directory listing? When you access `/index.php`, what happens?
It is just a directory listing.
When I access `http://127.0.1.1/4get/index.php` I see an empty page with this line at the beginning:
> load( "home.html", [ "server_short_description" => htmlspecialchars(config::SERVER_SHORT_DESCRIPTION), "banner" => $images[rand(0, count($images) - 1)] ] );
I installed this that I believe being the corresponding version:
```
pacman -Ql php-apache
php-apache /etc/
php-apache /etc/httpd/
php-apache /etc/httpd/conf/
php-apache /etc/httpd/conf/extra/
php-apache /etc/httpd/conf/extra/php_module.conf
php-apache /usr/
php-apache /usr/lib/
php-apache /usr/lib/httpd/
php-apache /usr/lib/httpd/modules/
php-apache /usr/lib/httpd/modules/libphp.so
```
I now added to the conf:
```
Include conf/extra/php_module.conf
LoadModule php_module /usr/lib/httpd/modules/libphp.so
```
It does not start anymore:
` Apache is running a threaded MPM, but your PHP Module is not compiled to be threadsafe. You need to recompile PHP.`
Do I need to recompile it or should I do something else?
Disabling mpm_event and enabling mpm_prefork makes apache starts but I see what I was seeing before.
```
#LoadModule mpm_event_module modules/mod_mpm_event.so
LoadModule mpm_prefork_module modules/mod_mpm_prefork.so
```
What the fuck kind of distro ships broken apache2 versions that can't run PHP? Don't recompile PHP, that's a whole new rabbit hole that I swear you do not want to get involved in. This shit is asking you to enable ZTS support, which is an absolute fucking pain in the ass to setup.
Your only option would be to try installing 4get through docker at this point. This is insane.
What the fuck kind of distro ships broken apache2 versions that can't run PHP? Don't recompile PHP, that's a whole new rabbit hole that I *swear* you do not want to get involved in. This shit is asking you to enable ZTS support, which is an absolute fucking pain in the ass to setup.
Your only option would be to try installing 4get through docker at this point. This is insane.
Since disabling mpm_event and enabling mpm_prefork does not have errors.
I'll search to how do it for Artix/Arch otherwise.
Does it need mpm_event therefore?
Since disabling mpm_event and enabling mpm_prefork does not have errors.
I'll search to how do it for Artix/Arch otherwise.
Dude I'm so sorry you're going through this, I have no fucking clue what mpm_event does. Shit just works on debian, and docker works on pretty much anything.
Let me know if you figure this out, I might write a guide for Artix users or something..
Dude I'm so sorry you're going through this, I have no fucking clue what `mpm_event` does. Shit just works on debian, and docker works on pretty much anything.
Let me know if you figure this out, I might write a guide for Artix users or something..
I did not look the Arch wiki after adding AddHandler php-script .php in the conf it starts.
I installed php83-curl and php83-mbstring from the AUR, those needed php83-8.3.7 too.
Not sure how that is supposed to work when php in repo is just php.
Now it shows up but it does not work, I attach screenshots.
I did not look the [Arch wiki]( https://wiki.archlinux.org/title/Apache_HTTP_Server#Using_libphp) after adding `AddHandler php-script .php` in the conf it starts.
I installed `php83-curl` and `php83-mbstring` from the AUR, those needed `php83-8.3.7` too.
Not sure how that is supposed to work when php in repo is just php.
Now it shows up but it does not work, I attach screenshots.
4get should be stored at the root of the webserver, otherwise you will see broken links.
/settings returning a 404 error means the .php internal redirect does not work.
4get should be stored at the root of the webserver, otherwise you will see broken links.
/settings returning a 404 error means the .php internal redirect does not work.
The home page is displayed better but settings, searching something and others still gets 404 error as before.
I moved the files from `/srv/http/4get` to `/srv/http/`.
And I updated the conf:
```
ServerRoot "/etc/httpd"
...
<Directory "/srv/http/">
...
```
The home page is displayed better but settings, searching something and others still gets 404 error as before.
<VirtualHost *:80>
# no domain = go to 4get.ca
RedirectMatch 301 ^(.*)$ https://4get.ca$1
</VirtualHost>
Not sure to how do that.
Is it similar to this?
```
<VirtualHost *:80>
# no domain = go to 4get.ca
RedirectMatch 301 ^(.*)$ https://4get.ca$1
</VirtualHost>
```
According to the apache wiki it is totally equivalent to a .htaccess file.
I am using this in the main config:
```
DocumentRoot "/srv/http/4get"
<Directory "/srv/http/4get">
Options +MultiViews
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^([^\.]+)$ $1.php [NC,L]
</Directory>
```
According to the apache wiki it is totally equivalent to a .htaccess file.
No in `/etc/httpd/conf/httpd.conf`.
I thought that SSL is not needed for local use and `/etc/httpd/confextra/httpd-ssl.conf` is not included anymore:
```
# Secure (SSL/TLS) connections
#Include conf/extra/httpd-ssl.conf
```
Another person got in touch with me, and also has trouble setting this up on Arch, so this isn't a you thing. I'll investigate later on and write a guide on how to fix this mess.
Another person got in touch with me, and also has trouble setting this up on Arch, so this isn't a you thing. I'll investigate later on and write a guide on how to fix this mess.
Chiming in, it's me the other person ;) I think that the best solution for Arch should be using a docker container. However, in every case, we have to face the same problem: if we selfhost locally on our machine, Im afraid that searching from http://localhost will happily serve our IP to the search engines, and that would obviously defeat the whole purpose of a private search. I have opened an issue to see whether it's possible to run a 4get private docker instance together with Tor.
Chiming in, it's me the other person ;) I think that the best solution for Arch should be using a docker container. However, in every case, we have to face the same problem: if we selfhost locally on our machine, Im afraid that searching from http://localhost will happily serve our IP to the search engines, and that would obviously defeat the whole purpose of a private search. I have opened an issue to see whether it's possible to run a 4get private docker instance together with Tor.
@root actually if you use Tor you don't need to run 4get on docker, you just have to edit two files in the data folder and 4get will route the traffic to the Tor proxy. If you, like me, run a docker container to get rid of the lack of dependencies, you'll have to add a Tor docker image to docker-compose.yml. But here you're on your own unless a guide will be ready, there's no official Tor docker image and the documentation on how to run Tor together with a search engine on docker is very sparse.
Also, keep in mind that running 4get over Tor or a vpn would have Google blocking you very soon, and there's no way to circumvent this.
@root actually if you use Tor you don't need to run 4get on docker, you just have to edit two files in the data folder and 4get will route the traffic to the Tor proxy. If you, like me, run a docker container to get rid of the lack of dependencies, you'll have to add a Tor docker image to docker-compose.yml. But here you're on your own unless a guide will be ready, there's no official Tor docker image and the documentation on how to run Tor together with a search engine on docker is very sparse.
Also, keep in mind that running 4get over Tor or a vpn would have Google blocking you very soon, and there's no way to circumvent this.
Also, keep in mind that running 4get over Tor or a vpn would have Google blocking you very soon, and there's no way to circumvent this.
You can setup proxychains
>Also, keep in mind that running 4get over Tor or a vpn would have Google blocking you very soon, and there's no way to circumvent this.
You can setup proxychains
Great, I have just found out a brand new docker-tor guide, amazing surprise and I immediately followed it, thank you guys! :) I wonder how would it be possible to use proxychains to launch the browser. The proxychains.conf has socks4 127.0.0.1 9050 as default proxy, it looks like it needs Tor running on the host. I wonder how this could work with 4get running over Tor in an isolated container.
Great, I have just found out a brand new docker-tor guide, amazing surprise and I immediately followed it, thank you guys! :) I wonder how would it be possible to use proxychains to launch the browser. The proxychains.conf has socks4 127.0.0.1 9050 as default proxy, it looks like it needs Tor running on the host. I wonder how this could work with 4get running over Tor in an isolated container.
Simply running docker compose up -d in 4get repo (/srv/http/4get here) works.
I can use 4get connecting to http://localhost/ and search.
After I tried to add tor.
I made the files in /srv/http/4get/data and /srv/http/4get/proxies.
I am not sure to how make the folders.
If I run docker compose up -d in .../4get/data and in .../4get/proxies 4get shows up but I cannot search.
Simply running `docker compose up -d` in 4get repo (`/srv/http/4get` here) works.
I can use 4get connecting to `http://localhost/` and search.
After I tried to add tor.
I made the files in `/srv/http/4get/data` and `/srv/http/4get/proxies`.
I am not sure to how make the folders.
If I run `docker compose up -d` in `.../4get/data` and in `.../4get/proxies` 4get shows up but I cannot search.
I think the easy way should be creating a 4get folder in your home directory and there create 1) a "proxies" folder with the suggested onion.txt file and 2) the docker-compose.yml file. Then cd to this 4get folder and run docker compose up -d. It works for me.
I think the easy way should be creating a 4get folder in your home directory and there create 1) a "proxies" folder with the suggested onion.txt file and 2) the docker-compose.yml file. Then cd to this 4get folder and run docker compose up -d. It works for me.
No, you have this 4get folder with two distinct things, a proxies subfolder and the docker compose file. Do you want to use your own torrc file ? The Tor container will already create a default torrc file.
No, you have this 4get folder with two distinct things, a proxies subfolder and the docker compose file. Do you want to use your own torrc file ? The Tor container will already create a default torrc file.
mount /path/4get/torrc:/etc/tor/torrc (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
Indeed /etc/tor/torrc is a text file and 4get/torrc is a folder.
I removed that line (- ./torrc/torrc:/etc/tor/torrc) and localhost is completely down.
It fails to start:
```
mount /path/4get/torrc:/etc/tor/torrc (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
```
Indeed `/etc/tor/torrc` is a text file and 4get/torrc is a folder.
I removed that line (`- ./torrc/torrc:/etc/tor/torrc`) and localhost is completely down.
Hmm weird, can you explain in details how did you setup the docker container? Have you removed your previous containers and images before starting this one?
Hmm weird, can you explain in details how did you setup the docker container? Have you removed your previous containers and images before starting this one?
No you don't need it, the docker compose file will do all the job for you. Luuul is the name of the docker repo with the two images (Tor and 4get). Remove all the images and containers that you have and start from scratch following exactly what the guide says.
No you don't need it, the docker compose file will do all the job for you. Luuul is the name of the docker repo with the two images (Tor and 4get). Remove all the images and containers that you have and start from scratch following exactly what the guide says.
Same error as before starting it creates a torrc folder.
The guide is not clear.
```
lsd --tree
./
├── docker-compose.yaml
└── proxies/
├── docker-compose.yaml
└── onion.txt
```
Like that?
Same error as before starting it creates a torrc folder.
No wait, remove the docker compose file from the proxies folder. You just need the first one. Just move to that lsd folder and type docker compose up -d
No wait, remove the docker compose file from the proxies folder. You just need the first one. Just move to that lsd folder and type docker compose up -d
There are two docker-compose.yaml in the guide I am confused.
I moved the one in proxies and as before I ran docker compose up -d in the parent folder.
Same error as before starting it creates a torrc folder.
The folder is named 4get lsd is a command: Modern ls with a lot of pretty colors and awesome icons.
There are two `docker-compose.yaml` in the guide I am confused.
I moved the one in proxies and as before I ran `docker compose up -d` in the parent folder.
Same error as before starting it creates a torrc folder.
The folder is named 4get `lsd` is a command: `Modern ls with a lot of pretty colors and awesome icons`.
Oh sorry, I have always just used ls :) the first docker compose file is to route searches over Tor, the second is to be found as a hidden service by Tor users, so I guess you would only want the first one as me
Oh sorry, I have always just used ls :) the first docker compose file is to route searches over Tor, the second is to be found as a hidden service by Tor users, so I guess you would only want the first one as me
Secure Connection Failed
An error occurred during a connection to localhost:80. SSL received a record that exceeded the maximum permissible length.
Error code: SSL_ERROR_RX_RECORD_TOO_LONG
There is no option to add a security exception to bypass this type of error.
I use librewolf and I have this error:
```
Secure Connection Failed
An error occurred during a connection to localhost:80. SSL received a record that exceeded the maximum permissible length.
Error code: SSL_ERROR_RX_RECORD_TOO_LONG
```
https://support.mozilla.org/en-US/kb/secure-connection-failed-firefox-did-not-connect?as=u#w_secure-connection-failed
>There is no option to add a security exception to bypass this type of error.
@root can you paste the output of docker container ls? And the content of the onion.txt in the proxies folder? It looks like something is still misconfigured.
@root can you paste the output of `docker container ls`? And the content of the onion.txt in the proxies folder? It looks like something is still misconfigured.
docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
acf0e6039afa luuul/4get:latest "./docker/docker-ent…" 40 seconds ago Up 39 seconds 0.0.0.0:80->80/tcp, :::80->80/tcp, 443/tcp 4get-fourget-1
1b579790132e luuul/tor:latest "/usr/bin/tor" 40 seconds ago Up 39 seconds (health: starting) 9050/tcp 4get-tor-1
88e8598b2706 luuul/4get:latest "./docker/docker-ent…" 25 hours ago Up 23 hours 80/tcp, 443/tcp data-fourget-1
cat proxies/onion.txt
# proxies/onion.txt
# Note: "tor" is the service name of luuul/tor in docker-compose.yaml
socks5:tor:9050::
```
docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
acf0e6039afa luuul/4get:latest "./docker/docker-ent…" 40 seconds ago Up 39 seconds 0.0.0.0:80->80/tcp, :::80->80/tcp, 443/tcp 4get-fourget-1
1b579790132e luuul/tor:latest "/usr/bin/tor" 40 seconds ago Up 39 seconds (health: starting) 9050/tcp 4get-tor-1
88e8598b2706 luuul/4get:latest "./docker/docker-ent…" 25 hours ago Up 23 hours 80/tcp, 443/tcp data-fourget-1
```
```
cat proxies/onion.txt
# proxies/onion.txt
# Note: "tor" is the service name of luuul/tor in docker-compose.yaml
socks5:tor:9050::
```
Shit
This scraper returned an error:
Failed to get html
does this error appear on every engine? If so, that might be an issue with your DNS resolver.
> I was not using http sorry.
>
> I have this error:
> ```
> Shit
> This scraper returned an error:
> Failed to get html
> ```
>
does this error appear on every engine? If so, that might be an issue with your DNS resolver.
@root you have two running docker 4get containers.Try to delete the last one with docker container rm -f 88e8598b2706. If this doesn't work, try to remove all the containers and images that you have and start all the setup from scratch.
@root you have two running docker 4get containers.Try to delete the last one with docker container rm -f 88e8598b2706. If this doesn't work, try to remove all the containers and images that you have and start all the setup from scratch.
@lolcat I tested it with 4get.ca and it works e.g. https://4get.ca/web?s=test.
Before and now too.
No I don't mean on 4get.ca, I mean on your own config. Do all of the scrapers fail?
> @lolcat I tested it with 4get.ca and it works e.g. `https://4get.ca/web?s=test`.
> Before and now too.
No I don't mean on 4get.ca, I mean on your own config. Do all of the scrapers fail?
No I don't mean on 4get.ca, I mean on your own config. Do all of the scrapers fail?
Also yandex, brave , curlie, marginalia and qwant.
In `/srv/http/4get/` I just ran `chmod 777 -R icons/`.
In my home in 4get I have only:
```
lsd --tree
./
├── docker-compose.yaml
└── proxies/
└── onion.txt
```
```
cat docker-compose.yaml
# docker-compose.yaml
version: "3.7"
...
depends_on:
- tor
volumes:
- ./proxies/:/srv/http/4get/data/proxies/
```
>No I don't mean on 4get.ca, I mean on your own config. Do all of the scrapers fail?
Also yandex, brave , curlie, marginalia and qwant.
docker rmi -f $(docker images -aq)
docker rmi -f $(docker images -aq)
docker system prune -a
After that still Failed to get html.
In /etc/tor/torrc I added SocksPort 0.0.0.0:9050.
```
docker rmi -f $(docker images -aq)
docker rmi -f $(docker images -aq)
docker system prune -a
```
After that still `Failed to get html`.
In `/etc/tor/torrc` I added `SocksPort 0.0.0.0:9050`.
It looks like you're mixing the steps from your previous install attempt and the steps from docker
please don't put SocksPort 0.0.0.0:9050 in your host's /etc/tor/torrc
This is good! The files you have in your home folder are all you need if you're using the docker + tor guide. However the contents in your docker compose seems incomplete,
Put the following in your docker-compose.yaml file in your home's 4get folder
then cd into that directory and run docker compose up -d
everything should work
If there are issues please give the result of these commands
docker ps -a
docker container ls -a
It looks like you're mixing the steps from your previous install attempt and the steps from docker
please don't put `SocksPort 0.0.0.0:9050` in your host's /etc/tor/torrc
> In my home in 4get I have only:
```
lsd --tree
./
├── docker-compose.yaml
└── proxies/
└── onion.txt
```
This is good! The files you have in your home folder are all you need if you're using the docker + tor guide. However the contents in your docker compose seems incomplete,
Put the following in your docker-compose.yaml file in your home's 4get folder
```
# docker-compose.yaml
version: "3.7"
services:
tor:
image: luuul/tor:latest
restart: unless-stopped
# Warning: Do not publish port 9050
fourget:
image: luuul/4get:latest
restart: unless-stopped
environment:
- FOURGET_PROTO=http
- FOURGET_SERVER_NAME=4get.ca
# loads proxies/onion.txt
- FOURGET_PROXY_DDG="onion"
- FOURGET_PROXY_BRAVE="onion"
- FOURGET_PROXY_FB="onion"
- FOURGET_PROXY_GOOGLE="onion"
- FOURGET_PROXY_QWANT="onion"
- FOURGET_PROXY_MARGINALIA="onion"
- FOURGET_PROXY_MOJEEK="onion"
- FOURGET_PROXY_SC="onion"
- FOURGET_PROXY_SPOTIFY="onion"
- FOURGET_PROXY_WIBY="onion"
- FOURGET_PROXY_CURLIE="onion"
- FOURGET_PROXY_YT="onion"
- FOURGET_PROXY_YEP="onion"
- FOURGET_PROXY_PINTEREST="onion"
- FOURGET_PROXY_SEZNAM="onion"
- FOURGET_PROXY_NAVER="onion"
- FOURGET_PROXY_GREPPR="onion"
- FOURGET_PROXY_CROWDVIEW="onion"
- FOURGET_PROXY_MWMBL="onion"
- FOURGET_PROXY_FTM="onion"
- FOURGET_PROXY_IMGUR="onion"
- FOURGET_PROXY_YANDEX_W="onion"
- FOURGET_PROXY_YANDEX_I="onion"
- FOURGET_PROXY_YANDEX_V="onion"
ports:
- "80:80"
depends_on:
- tor
volumes:
- ./proxies/:/var/www/html/4get/data/proxies/
```
then cd into that directory and run docker compose up -d
everything should work
If there are issues please give the result of these commands
```
docker ps -a
docker container ls -a
```
@Evhorizon I think that I did not change network settings.
I just tried firefox same there.
@throwaway Shouldn't I change - ./proxies/:/var/www/html/4get/data/proxies to - ./proxies/:/srv/http/4get/data/proxies/ since 4get repo is there?
I put only the beginning and the part that I changed with ... in between in that file.
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
eaa119314b01 luuul/4get:latest "./docker/docker-ent…" 11 seconds ago Up 10 seconds 0.0.0.0:80->80/tcp, :::80->80/tcp, 443/tcp 4get-fourget-1
7ad8dfbb4b06 luuul/tor:latest "/usr/bin/tor" 11 seconds ago Up 10 seconds (health: starting) 9050/tcp
docker container ls -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
eaa119314b01 luuul/4get:latest "./docker/docker-ent…" 27 seconds ago Up 26 seconds 0.0.0.0:80->80/tcp, :::80->80/tcp, 443/tcp 4get-fourget-1
7ad8dfbb4b06 luuul/tor:latest "/usr/bin/tor" 27 seconds ago Up 26 seconds (health: starting) 9050/tcp
I reset /etc/tor/torrc no need to run tor service?
With it running or not curlie and DDG scraper: Failed to get html.
@Evhorizon I think that I did not change network settings.
I just tried `firefox` same there.
@throwaway Shouldn't I change `- ./proxies/:/var/www/html/4get/data/proxies` to `- ./proxies/:/srv/http/4get/data/proxies/` since 4get repo is there?
I put only the beginning and the part that I changed with `...` in between in that file.
```
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
eaa119314b01 luuul/4get:latest "./docker/docker-ent…" 11 seconds ago Up 10 seconds 0.0.0.0:80->80/tcp, :::80->80/tcp, 443/tcp 4get-fourget-1
7ad8dfbb4b06 luuul/tor:latest "/usr/bin/tor" 11 seconds ago Up 10 seconds (health: starting) 9050/tcp
docker container ls -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
eaa119314b01 luuul/4get:latest "./docker/docker-ent…" 27 seconds ago Up 26 seconds 0.0.0.0:80->80/tcp, :::80->80/tcp, 443/tcp 4get-fourget-1
7ad8dfbb4b06 luuul/tor:latest "/usr/bin/tor" 27 seconds ago Up 26 seconds (health: starting) 9050/tcp
```
I reset `/etc/tor/torrc` no need to run tor service?
With it running or not curlie and DDG scraper: `Failed to get html`.
Shouldn't I change - ./proxies/:/var/www/html/4get/data/proxies to - ./proxies/:/srv/http/4get/data/proxies/ since 4get repo is there?
In docker, volumes are mounted from the host machine to the container
the 4get container always reads from /var/www/html/4get/data/proxies so that part will not change
the ./proxies/ is relative to where you're running docker compose. If you'd like you can change this part to /srv/http/4get/data/proxies/
but since your proxy list onion.txt is in the 4get folder in your home directory you don't need to do this.
I reset /etc/tor/torrc no need to run tor service?
Yes. The tor service will run inside the tor container luuul/tor which has a working torrc inside
I put only the beginning and the part that I changed with ... in between in that file.
Okay. Please make sure that your docker-compose.yaml contains only the content I mentioned previously
Docker compose will restart already existing containers if they exist. And containers contain their own state. We should try starting with a blank state and recreate our containers.
stop all containers with docker container stop $(docker container ls -aq)
remove containers with docker container rm $(docker container ls -aq)
then try docker compose up which will recreate the containers
> Shouldn't I change - ./proxies/:/var/www/html/4get/data/proxies to - ./proxies/:/srv/http/4get/data/proxies/ since 4get repo is there?
In docker, volumes are mounted from the host machine to the container
the 4get container always reads from /var/www/html/4get/data/proxies so that part will not change
the `./proxies/` is relative to where you're running docker compose. If you'd like you can change this part to `/srv/http/4get/data/proxies/`
so the volume mount could look like
```
- /srv/http/4get/data/proxies/:/var/www/html/4get/data/proxies
```
but since your proxy list `onion.txt` is in the 4get folder in your home directory you don't need to do this.
>I reset /etc/tor/torrc no need to run tor service?
Yes. The tor service will run inside the tor container `luuul/tor` which has a working torrc inside
>I put only the beginning and the part that I changed with ... in between in that file.
Okay. Please make sure that your docker-compose.yaml contains only the content I mentioned previously
---
Docker compose will restart already existing containers if they exist. And containers contain their own state. We should try starting with a blank state and recreate our containers.
stop all containers with ` docker container stop $(docker container ls -aq)`
remove containers with `docker container rm $(docker container ls -aq)`
then try docker compose up which will recreate the containers
I only changed FOURGET_SERVER_NAME so that I can distinguish 4get.ca and the local instance when I look the search engines in the browser.
The cloned 4get repo in `/srv/http/` is totally pointless so.
It works thanks.
I could make a version of https://git.lolcat.ca/lolcat/4get/src/branch/master/docs/docker_tor.md for local use.
I only changed `FOURGET_SERVER_NAME` so that I can distinguish 4get.ca and the local instance when I look the search engines in the browser.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
I am looking the guide https://git.lolcat.ca/lolcat/4get/src/branch/master/docs/apache2.md.
Artix instructions should be the same for Arch (except the service commands).
I did the step before https://git.lolcat.ca/lolcat/4get/src/branch/master/docs/apache2.md#default-sslconf but I may have missed something.
/etc/httpd/confextra/httpd-ssl.conf:I enabled the file commenting out
Include conf/extra/httpd-ssl.confin/etc/httpd/conf/httpd.confI had errors about
DEFLATEso I commented it.Then I commented also
SSLCertificateFileandSSLCertificateFilebecause I do not have them yet.Apache won't start and I do not see anymore errors in the log.
This is how I cloned the repo.
I am not sure if I specified
DocumentRoot /srv/http/c,RedirectMatch 301 ^(.*)$ https://c.ca$1etc. correctly.Is it fine asking here?
Asking here is fine, lol
For the deflate error, please try enabling the extension using
sudo a2enmod enable deflateor something like that. Also try withmod-deflateif that doesn't work. If the module doesn't exist, try and look in your package manager for apache2 package extras.That
RedirectMatchdirective will redirect all traffic on port 443 (https, assumingSSLEngineis on) to whatever domain you specify. I'm not sure what you're trying to do here, are you trying to only allow traffic throughlocalhost? If so, leave out the ssl config stuff. If not, try and fix it like this, assuming you're using thec.cadomain:RedirectMatch 301 ^(.*)$ https://c.ca/$1Again, this will redirect all SSL traffic to your c.ca domain, so make sure that this is what you want.
Now, Apache2 won't start because there is no
SSLCertificateFilewhileSSLEngineis on. Generate your certificates and uncomment theSSLCertificateFiledirectives, all the while making sure you supply the right file paths.Also, not quite sure why you have 2 rulesets with
ServerName c.ca? It's better to merge them to avoid conflicts.Keep me posted!
I'd like to set up my 4get instance that I can use locally so I supposed that you use
https://http://localhost/c.cain these cases.I am not sure to how set up the local instance.
I made 2 rulesets trying to following the guide but I am not familiar with this.
In
/etc/httpd/conf/httpd.confI now commented outLoadModule deflate_module modules/mod_deflate.so, looks like thata2enmodis Debian specific.In this case do I need ssl config?
You do not need an ssl config for local use. Why do you insist on using
https://http://localhost/c.ca? This is wrong, you specify the protocol 2 times. Since you're running this locally, there is no need to setup redirects, you can just access your webserver directly with http://localhost .Please let me know if you have any other questions :)
Do I need to use
/etc/httpd/conf/extra/httpd-vhosts.conf?How do I make 4get instance visible on localhost?
It should be visible by default. The only thing you need to do is the url's .php extension internal redirect (the
RewriteRuledirectives)Could you make an example of all the things that I need?
Not sure where put those rules in
/etc/httpd/conf/httpd.conf?Or
At this point, what I would suggest doing would be to run
dpkg-reconfigureon your apache2 webserver and create an.htaccessconfiguration file in your/var/www/4get/folder that can get rid of the.phpextension by doing an internal URL rewrite. This is all you need to do in order to make 4get work locally.You can look up how to enable .htaccess online and then just place the following rules in there:
When you make configuration changes, make sure to restart apache2 every time.
Please let me know if you encounter any other issues.
https://httpd.apache.org/docs/current/howto/htaccess.html#when
I made this in the conf and I also made the equivalent
.htaccessfile in/srv/http/4get./srv/httpis the defautl value in the conf, and I enabledAllowOverride:When I go to localhost I see the 4get folder and I can browse it but I do not know to how use it as search engine.
For local use do I need all the packages
php-mbstring apache2 certbot php-imagick imagemagick php-curl curl php-apcu git libapache2-mod-php?And ssl, rewrite, proxy and headers modules?
You need
php-mbstring apache2 php-imagick imagemagick php-curl curl php-apcu libapache2-mod-phpand rewrite modules to get rid of the .php extension.Right now I do not have
php-mbstringandphp-curl.Is it supposed to show up at least?
I don't really understand why you're having this many issues. I have followed these installation steps myself multiple times now while migrating machines and I can assure you that they work.
When you access your webserver, does anything render, or is it just a directory listing? When you access
/index.php, what happens?It is just a directory listing.
When I access
http://127.0.1.1/4get/index.phpI see an empty page with this line at the beginning:Did you install
libapache2-mod-php?I installed this that I believe being the corresponding version:
I now added to the conf:
It does not start anymore:
Apache is running a threaded MPM, but your PHP Module is not compiled to be threadsafe. You need to recompile PHP.Do I need to recompile it or should I do something else?
Disabling mpm_event and enabling mpm_prefork makes apache starts but I see what I was seeing before.
What the fuck kind of distro ships broken apache2 versions that can't run PHP? Don't recompile PHP, that's a whole new rabbit hole that I swear you do not want to get involved in. This shit is asking you to enable ZTS support, which is an absolute fucking pain in the ass to setup.
Your only option would be to try installing 4get through docker at this point. This is insane.
Does it need mpm_event therefore?
Since disabling mpm_event and enabling mpm_prefork does not have errors.
I'll search to how do it for Artix/Arch otherwise.
Dude I'm so sorry you're going through this, I have no fucking clue what
mpm_eventdoes. Shit just works on debian, and docker works on pretty much anything.Let me know if you figure this out, I might write a guide for Artix users or something..
I did not look the Arch wiki after adding
AddHandler php-script .phpin the conf it starts.I installed
php83-curlandphp83-mbstringfrom the AUR, those neededphp83-8.3.7too.Not sure how that is supposed to work when php in repo is just php.
Now it shows up but it does not work, I attach screenshots.
4get should be stored at the root of the webserver, otherwise you will see broken links.
/settings returning a 404 error means the .php internal redirect does not work.
I moved the files from
/srv/http/4getto/srv/http/.And I updated the conf:
The home page is displayed better but settings, searching something and others still gets 404 error as before.
Setup the RedirectRule config
Not sure to how do that.
Is it similar to this?
No, you use the rules here
#19 (comment)
I am using this in the main config:
According to the apache wiki it is totally equivalent to a .htaccess file.
Place this under a
<VirtualHost *:80>in your configuration and restart apacheI added it and as before I see
Error 404.Nothing new in the log.
You placed this in
/etc/httpd/confextra/httpd-ssl.conf?No in
/etc/httpd/conf/httpd.conf.I thought that SSL is not needed for local use and
/etc/httpd/confextra/httpd-ssl.confis not included anymore:Another person got in touch with me, and also has trouble setting this up on Arch, so this isn't a you thing. I'll investigate later on and write a guide on how to fix this mess.
Chiming in, it's me the other person ;) I think that the best solution for Arch should be using a docker container. However, in every case, we have to face the same problem: if we selfhost locally on our machine, Im afraid that searching from http://localhost will happily serve our IP to the search engines, and that would obviously defeat the whole purpose of a private search. I have opened an issue to see whether it's possible to run a 4get private docker instance together with Tor.
Possibly I'd run it without a docker but if you need it for Tor or something I guess I'll use it.
I will wait for the guide then, thanks.
@root actually if you use Tor you don't need to run 4get on docker, you just have to edit two files in the data folder and 4get will route the traffic to the Tor proxy. If you, like me, run a docker container to get rid of the lack of dependencies, you'll have to add a Tor docker image to docker-compose.yml. But here you're on your own unless a guide will be ready, there's no official Tor docker image and the documentation on how to run Tor together with a search engine on docker is very sparse.
Also, keep in mind that running 4get over Tor or a vpn would have Google blocking you very soon, and there's no way to circumvent this.
You can setup proxychains
Great, I have just found out a brand new docker-tor guide, amazing surprise and I immediately followed it, thank you guys! :) I wonder how would it be possible to use proxychains to launch the browser. The proxychains.conf has socks4 127.0.0.1 9050 as default proxy, it looks like it needs Tor running on the host. I wonder how this could work with 4get running over Tor in an isolated container.
Usually I scrape DuckDuckGO so I think that I can just use docker with tor.
When/if somebody makes a guide can you let me know?
Hey @root the guide is on
https://git.lolcat.ca/lolcat/4get/src/branch/master/docs/docker_tor.md
Thanks @Evhorizon.
Do I need to follow https://git.lolcat.ca/lolcat/4get/src/branch/master/docs/docker.md before and than that?
@root yes
Simply running
docker compose up -din 4get repo (/srv/http/4gethere) works.I can use 4get connecting to
http://localhost/and search.After I tried to add tor.
I made the files in
/srv/http/4get/dataand/srv/http/4get/proxies.I am not sure to how make the folders.
If I run
docker compose up -din.../4get/dataand in.../4get/proxies4get shows up but I cannot search.I think the easy way should be creating a 4get folder in your home directory and there create 1) a "proxies" folder with the suggested onion.txt file and 2) the docker-compose.yml file. Then cd to this 4get folder and run docker compose up -d. It works for me.
In this 4get folder is
docker-compose.yamlinside proxies folder?Where do I put torrc and other
docker-compose.yamlin/srv/http/4get?No, you have this 4get folder with two distinct things, a proxies subfolder and the docker compose file. Do you want to use your own torrc file ? The Tor container will already create a default torrc file.
No, I do not have any torrc file and I do not have experience with this.
Where do I create the torrc file?
Ok so you don't need to create it, just run docker compose and everything will work out of the box
It fails to start:
Indeed
/etc/tor/torrcis a text file and 4get/torrc is a folder.I removed that line (
- ./torrc/torrc:/etc/tor/torrc) and localhost is completely down.Hmm weird, can you explain in details how did you setup the docker container? Have you removed your previous containers and images before starting this one?
Do I need to download the images manually? What's
luuul?No you don't need it, the docker compose file will do all the job for you. Luuul is the name of the docker repo with the two images (Tor and 4get). Remove all the images and containers that you have and start from scratch following exactly what the guide says.
The guide is not clear.
Like that?
Same error as before starting it creates a torrc folder.
No wait, remove the docker compose file from the proxies folder. You just need the first one. Just move to that lsd folder and type docker compose up -d
There are two
docker-compose.yamlin the guide I am confused.I moved the one in proxies and as before I ran
docker compose up -din the parent folder.Same error as before starting it creates a torrc folder.
The folder is named 4get
lsdis a command:Modern ls with a lot of pretty colors and awesome icons.Oh sorry, I have always just used ls :) the first docker compose file is to route searches over Tor, the second is to be found as a hidden service by Tor users, so I guess you would only want the first one as me
I use librewolf and I have this error:
https://support.mozilla.org/en-US/kb/secure-connection-failed-firefox-did-not-connect?as=u#w_secure-connection-failed
Why did you setup SSL over localhost?
I do not know why I have it.
How can I disable it?
@root dumb question, are you sure you are typing http://localhost/ and not https://localhost/ ?
I was not using http sorry.
I have this error:
@root can you paste the output of
docker container ls? And the content of the onion.txt in the proxies folder? It looks like something is still misconfigured.does this error appear on every engine? If so, that might be an issue with your DNS resolver.
@lolcat I tested it with 4get.ca and it works e.g.
https://4get.ca/web?s=test.Before and now too.
@root you have two running docker 4get containers.Try to delete the last one with docker container rm -f 88e8598b2706. If this doesn't work, try to remove all the containers and images that you have and start all the setup from scratch.
No I don't mean on 4get.ca, I mean on your own config. Do all of the scrapers fail?
In
/srv/http/4get/I just ranchmod 777 -R icons/.In my home in 4get I have only:
Also yandex, brave , curlie, marginalia and qwant.
After that still
Failed to get html.In
/etc/tor/torrcI addedSocksPort 0.0.0.0:9050.@root Just wonder, how are your librewolf network settings?
It looks like you're mixing the steps from your previous install attempt and the steps from docker
please don't put
SocksPort 0.0.0.0:9050in your host's /etc/tor/torrcThis is good! The files you have in your home folder are all you need if you're using the docker + tor guide. However the contents in your docker compose seems incomplete,
Put the following in your docker-compose.yaml file in your home's 4get folder
then cd into that directory and run docker compose up -d
everything should work
If there are issues please give the result of these commands
@Evhorizon I think that I did not change network settings.
I just tried
firefoxsame there.@throwaway Shouldn't I change
- ./proxies/:/var/www/html/4get/data/proxiesto- ./proxies/:/srv/http/4get/data/proxies/since 4get repo is there?I put only the beginning and the part that I changed with
...in between in that file.I reset
/etc/tor/torrcno need to run tor service?With it running or not curlie and DDG scraper:
Failed to get html.In docker, volumes are mounted from the host machine to the container
the 4get container always reads from /var/www/html/4get/data/proxies so that part will not change
the
./proxies/is relative to where you're running docker compose. If you'd like you can change this part to/srv/http/4get/data/proxies/so the volume mount could look like
but since your proxy list
onion.txtis in the 4get folder in your home directory you don't need to do this.Yes. The tor service will run inside the tor container
luuul/torwhich has a working torrc insideOkay. Please make sure that your docker-compose.yaml contains only the content I mentioned previously
Docker compose will restart already existing containers if they exist. And containers contain their own state. We should try starting with a blank state and recreate our containers.
stop all containers with
docker container stop $(docker container ls -aq)remove containers with
docker container rm $(docker container ls -aq)then try docker compose up which will recreate the containers
The cloned 4get repo in
/srv/http/is totally pointless so.It works thanks.
I could make a version of https://git.lolcat.ca/lolcat/4get/src/branch/master/docs/docker_tor.md for local use.
I only changed
FOURGET_SERVER_NAMEso that I can distinguish 4get.ca and the local instance when I look the search engines in the browser.