Question: how set 4get instance in Artix #19
Loading…
Reference in New Issue
No description provided.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
I am looking the guide https://git.lolcat.ca/lolcat/4get/src/branch/master/docs/apache2.md.
Artix instructions should be the same for Arch (except the service commands).
I did the step before https://git.lolcat.ca/lolcat/4get/src/branch/master/docs/apache2.md#default-sslconf but I may have missed something.
/etc/httpd/confextra/httpd-ssl.conf
:I enabled the file commenting out
Include conf/extra/httpd-ssl.conf
in/etc/httpd/conf/httpd.conf
I had errors about
DEFLATE
so I commented it.Then I commented also
SSLCertificateFile
andSSLCertificateFile
because I do not have them yet.Apache won't start and I do not see anymore errors in the log.
This is how I cloned the repo.
I am not sure if I specified
DocumentRoot /srv/http/c
,RedirectMatch 301 ^(.*)$ https://c.ca$1
etc. correctly.Is it fine asking here?
Asking here is fine, lol
For the deflate error, please try enabling the extension using
sudo a2enmod enable deflate
or something like that. Also try withmod-deflate
if that doesn't work. If the module doesn't exist, try and look in your package manager for apache2 package extras.That
RedirectMatch
directive will redirect all traffic on port 443 (https, assumingSSLEngine
is on) to whatever domain you specify. I'm not sure what you're trying to do here, are you trying to only allow traffic throughlocalhost
? If so, leave out the ssl config stuff. If not, try and fix it like this, assuming you're using thec.ca
domain:RedirectMatch 301 ^(.*)$ https://c.ca/$1
Again, this will redirect all SSL traffic to your c.ca domain, so make sure that this is what you want.
Now, Apache2 won't start because there is no
SSLCertificateFile
whileSSLEngine
is on. Generate your certificates and uncomment theSSLCertificateFile
directives, all the while making sure you supply the right file paths.Also, not quite sure why you have 2 rulesets with
ServerName c.ca
? It's better to merge them to avoid conflicts.Keep me posted!
I'd like to set up my 4get instance that I can use locally so I supposed that you use
https://http://localhost/c.ca
in these cases.I am not sure to how set up the local instance.
I made 2 rulesets trying to following the guide but I am not familiar with this.
In
/etc/httpd/conf/httpd.conf
I now commented outLoadModule deflate_module modules/mod_deflate.so
, looks like thata2enmod
is Debian specific.In this case do I need ssl config?
You do not need an ssl config for local use. Why do you insist on using
https://http://localhost/c.ca
? This is wrong, you specify the protocol 2 times. Since you're running this locally, there is no need to setup redirects, you can just access your webserver directly with http://localhost .Please let me know if you have any other questions :)
Do I need to use
/etc/httpd/conf/extra/httpd-vhosts.conf
?How do I make 4get instance visible on localhost?
It should be visible by default. The only thing you need to do is the url's .php extension internal redirect (the
RewriteRule
directives)Could you make an example of all the things that I need?
Not sure where put those rules in
/etc/httpd/conf/httpd.conf
?Or
At this point, what I would suggest doing would be to run
dpkg-reconfigure
on your apache2 webserver and create an.htaccess
configuration file in your/var/www/4get/
folder that can get rid of the.php
extension by doing an internal URL rewrite. This is all you need to do in order to make 4get work locally.You can look up how to enable .htaccess online and then just place the following rules in there:
When you make configuration changes, make sure to restart apache2 every time.
Please let me know if you encounter any other issues.
https://httpd.apache.org/docs/current/howto/htaccess.html#when
I made this in the conf and I also made the equivalent
.htaccess
file in/srv/http/4get
./srv/http
is the defautl value in the conf, and I enabledAllowOverride
:When I go to localhost I see the 4get folder and I can browse it but I do not know to how use it as search engine.
For local use do I need all the packages
php-mbstring apache2 certbot php-imagick imagemagick php-curl curl php-apcu git libapache2-mod-php
?And ssl, rewrite, proxy and headers modules?
You need
php-mbstring apache2 php-imagick imagemagick php-curl curl php-apcu libapache2-mod-php
and rewrite modules to get rid of the .php extension.Right now I do not have
php-mbstring
andphp-curl
.Is it supposed to show up at least?
I don't really understand why you're having this many issues. I have followed these installation steps myself multiple times now while migrating machines and I can assure you that they work.
When you access your webserver, does anything render, or is it just a directory listing? When you access
/index.php
, what happens?It is just a directory listing.
When I access
http://127.0.1.1/4get/index.php
I see an empty page with this line at the beginning:Did you install
libapache2-mod-php
?I installed this that I believe being the corresponding version:
I now added to the conf:
It does not start anymore:
Apache is running a threaded MPM, but your PHP Module is not compiled to be threadsafe. You need to recompile PHP.
Do I need to recompile it or should I do something else?
Disabling mpm_event and enabling mpm_prefork makes apache starts but I see what I was seeing before.
What the fuck kind of distro ships broken apache2 versions that can't run PHP? Don't recompile PHP, that's a whole new rabbit hole that I swear you do not want to get involved in. This shit is asking you to enable ZTS support, which is an absolute fucking pain in the ass to setup.
Your only option would be to try installing 4get through docker at this point. This is insane.
Does it need mpm_event therefore?
Since disabling mpm_event and enabling mpm_prefork does not have errors.
I'll search to how do it for Artix/Arch otherwise.
Dude I'm so sorry you're going through this, I have no fucking clue what
mpm_event
does. Shit just works on debian, and docker works on pretty much anything.Let me know if you figure this out, I might write a guide for Artix users or something..
I did not look the Arch wiki after adding
AddHandler php-script .php
in the conf it starts.I installed
php83-curl
andphp83-mbstring
from the AUR, those neededphp83-8.3.7
too.Not sure how that is supposed to work when php in repo is just php.
Now it shows up but it does not work, I attach screenshots.
4get should be stored at the root of the webserver, otherwise you will see broken links.
/settings returning a 404 error means the .php internal redirect does not work.
I moved the files from
/srv/http/4get
to/srv/http/
.And I updated the conf:
The home page is displayed better but settings, searching something and others still gets 404 error as before.
Setup the RedirectRule config
Not sure to how do that.
Is it similar to this?
No, you use the rules here
#19 (comment)
I am using this in the main config:
According to the apache wiki it is totally equivalent to a .htaccess file.
Place this under a
<VirtualHost *:80>
in your configuration and restart apacheI added it and as before I see
Error 404
.Nothing new in the log.
You placed this in
/etc/httpd/confextra/httpd-ssl.conf
?No in
/etc/httpd/conf/httpd.conf
.I thought that SSL is not needed for local use and
/etc/httpd/confextra/httpd-ssl.conf
is not included anymore:Another person got in touch with me, and also has trouble setting this up on Arch, so this isn't a you thing. I'll investigate later on and write a guide on how to fix this mess.
Chiming in, it's me the other person ;) I think that the best solution for Arch should be using a docker container. However, in every case, we have to face the same problem: if we selfhost locally on our machine, Im afraid that searching from http://localhost will happily serve our IP to the search engines, and that would obviously defeat the whole purpose of a private search. I have opened an issue to see whether it's possible to run a 4get private docker instance together with Tor.
Possibly I'd run it without a docker but if you need it for Tor or something I guess I'll use it.
I will wait for the guide then, thanks.
@root actually if you use Tor you don't need to run 4get on docker, you just have to edit two files in the data folder and 4get will route the traffic to the Tor proxy. If you, like me, run a docker container to get rid of the lack of dependencies, you'll have to add a Tor docker image to docker-compose.yml. But here you're on your own unless a guide will be ready, there's no official Tor docker image and the documentation on how to run Tor together with a search engine on docker is very sparse.
Also, keep in mind that running 4get over Tor or a vpn would have Google blocking you very soon, and there's no way to circumvent this.
You can setup proxychains
Great, I have just found out a brand new docker-tor guide, amazing surprise and I immediately followed it, thank you guys! :) I wonder how would it be possible to use proxychains to launch the browser. The proxychains.conf has socks4 127.0.0.1 9050 as default proxy, it looks like it needs Tor running on the host. I wonder how this could work with 4get running over Tor in an isolated container.
Usually I scrape DuckDuckGO so I think that I can just use docker with tor.
When/if somebody makes a guide can you let me know?
Hey @root the guide is on
https://git.lolcat.ca/lolcat/4get/src/branch/master/docs/docker_tor.md
Thanks @Evhorizon.
Do I need to follow https://git.lolcat.ca/lolcat/4get/src/branch/master/docs/docker.md before and than that?
@root yes
Simply running
docker compose up -d
in 4get repo (/srv/http/4get
here) works.I can use 4get connecting to
http://localhost/
and search.After I tried to add tor.
I made the files in
/srv/http/4get/data
and/srv/http/4get/proxies
.I am not sure to how make the folders.
If I run
docker compose up -d
in.../4get/data
and in.../4get/proxies
4get shows up but I cannot search.I think the easy way should be creating a 4get folder in your home directory and there create 1) a "proxies" folder with the suggested onion.txt file and 2) the docker-compose.yml file. Then cd to this 4get folder and run docker compose up -d. It works for me.
In this 4get folder is
docker-compose.yaml
inside proxies folder?Where do I put torrc and other
docker-compose.yaml
in/srv/http/4get
?No, you have this 4get folder with two distinct things, a proxies subfolder and the docker compose file. Do you want to use your own torrc file ? The Tor container will already create a default torrc file.
No, I do not have any torrc file and I do not have experience with this.
Where do I create the torrc file?
Ok so you don't need to create it, just run docker compose and everything will work out of the box
It fails to start:
Indeed
/etc/tor/torrc
is a text file and 4get/torrc is a folder.I removed that line (
- ./torrc/torrc:/etc/tor/torrc
) and localhost is completely down.Hmm weird, can you explain in details how did you setup the docker container? Have you removed your previous containers and images before starting this one?
Do I need to download the images manually? What's
luuul
?No you don't need it, the docker compose file will do all the job for you. Luuul is the name of the docker repo with the two images (Tor and 4get). Remove all the images and containers that you have and start from scratch following exactly what the guide says.
The guide is not clear.
Like that?
Same error as before starting it creates a torrc folder.
No wait, remove the docker compose file from the proxies folder. You just need the first one. Just move to that lsd folder and type docker compose up -d
There are two
docker-compose.yaml
in the guide I am confused.I moved the one in proxies and as before I ran
docker compose up -d
in the parent folder.Same error as before starting it creates a torrc folder.
The folder is named 4get
lsd
is a command:Modern ls with a lot of pretty colors and awesome icons
.Oh sorry, I have always just used ls :) the first docker compose file is to route searches over Tor, the second is to be found as a hidden service by Tor users, so I guess you would only want the first one as me
I use librewolf and I have this error:
https://support.mozilla.org/en-US/kb/secure-connection-failed-firefox-did-not-connect?as=u#w_secure-connection-failed
Why did you setup SSL over localhost?
I do not know why I have it.
How can I disable it?
@root dumb question, are you sure you are typing http://localhost/ and not https://localhost/ ?
I was not using http sorry.
I have this error:
@root can you paste the output of
docker container ls
? And the content of the onion.txt in the proxies folder? It looks like something is still misconfigured.does this error appear on every engine? If so, that might be an issue with your DNS resolver.
@lolcat I tested it with 4get.ca and it works e.g.
https://4get.ca/web?s=test
.Before and now too.
@root you have two running docker 4get containers.Try to delete the last one with docker container rm -f 88e8598b2706. If this doesn't work, try to remove all the containers and images that you have and start all the setup from scratch.
No I don't mean on 4get.ca, I mean on your own config. Do all of the scrapers fail?
In
/srv/http/4get/
I just ranchmod 777 -R icons/
.In my home in 4get I have only:
Also yandex, brave , curlie, marginalia and qwant.
After that still
Failed to get html
.In
/etc/tor/torrc
I addedSocksPort 0.0.0.0:9050
.@root Just wonder, how are your librewolf network settings?
It looks like you're mixing the steps from your previous install attempt and the steps from docker
please don't put
SocksPort 0.0.0.0:9050
in your host's /etc/tor/torrcThis is good! The files you have in your home folder are all you need if you're using the docker + tor guide. However the contents in your docker compose seems incomplete,
Put the following in your docker-compose.yaml file in your home's 4get folder
then cd into that directory and run docker compose up -d
everything should work
If there are issues please give the result of these commands
@Evhorizon I think that I did not change network settings.
I just tried
firefox
same there.@throwaway Shouldn't I change
- ./proxies/:/var/www/html/4get/data/proxies
to- ./proxies/:/srv/http/4get/data/proxies/
since 4get repo is there?I put only the beginning and the part that I changed with
...
in between in that file.I reset
/etc/tor/torrc
no need to run tor service?With it running or not curlie and DDG scraper:
Failed to get html
.In docker, volumes are mounted from the host machine to the container
the 4get container always reads from /var/www/html/4get/data/proxies so that part will not change
the
./proxies/
is relative to where you're running docker compose. If you'd like you can change this part to/srv/http/4get/data/proxies/
so the volume mount could look like
but since your proxy list
onion.txt
is in the 4get folder in your home directory you don't need to do this.Yes. The tor service will run inside the tor container
luuul/tor
which has a working torrc insideOkay. Please make sure that your docker-compose.yaml contains only the content I mentioned previously
Docker compose will restart already existing containers if they exist. And containers contain their own state. We should try starting with a blank state and recreate our containers.
stop all containers with
docker container stop $(docker container ls -aq)
remove containers with
docker container rm $(docker container ls -aq)
then try docker compose up which will recreate the containers
The cloned 4get repo in
/srv/http/
is totally pointless so.It works thanks.
I could make a version of https://git.lolcat.ca/lolcat/4get/src/branch/master/docs/docker_tor.md for local use.
I only changed
FOURGET_SERVER_NAME
so that I can distinguish 4get.ca and the local instance when I look the search engines in the browser.