I'm a Linux/UNIX hack, so I play around a lot with shell scripts on my Epic, as well as just doing many things from ConnectBot and adb.
Everything's great from ConnectBot -- the emulator supports ANSI escape sequences, so it works perfectly with vi, command-line editing, etc. Great on the go!
However, I'd like to do the same thing when connect via 'adb shell' from my PC, and that's where it all goes to crap. The DOS command prompt window doesn't seem to support ANSI, window size is not passed through adb, so vi is unusable. Oddly, command-line editing seems to work. Also, I seem to be getting every command echoed, creating a spurious line.
Bottom line: I want to adb a local shell on my Epic from my PC and be able to use it the same way as I do on the phone itself via connectbot, yet with the obvious advantage of a big screen and normal keyboard. Anyone have any solution to this?
Thanks!!
QuickSSHd with PuTTy would probably get you what you're looking for, with the added benefit of being wireless.
I'm not sure where the limitation lies, but cygwin + adb might work.
Setup adb to use telnet and forward a local port, then you can use PuTTY to telnet into the local port. Details in this thread:
http://forum.xda-developers.com/showthread.php?t=535014
curvatura99 said:
Setup adb to use telnet and forward a local port, then you can use PuTTY to telnet into the local port. Details in this thread:
http://forum.xda-developers.com/showthread.php?t=535014
Click to expand...
Click to collapse
Oh, so close!! The thread above has the answer. In short, have adb listen to a local port on the Windows machine and forward it to the device, start the telnet server, and then use PuTTY to telnet into the device (from a Windows cmd prompt):
> adb forward tcp:9999 tcp:23
> adb shell telnetd
adb must be running as root.
Then, use PuTTY to telnet to localhost:9999 (the Windows machine @ 127.0.0.1, port 9999).
Almost works, except the telnet daemon on my Epic is refusing to connect! Looks like a security thing or something. Basically, I get the age-old UNIX telnet daemon message,
Entering character mode
Escape character is '^]'but instead of a "login:" prompt, the connection is immediately closed with the message,
Connection closed by foreign hostThis happens if I try to telnet locally to 127.0.0.1:23 on the Epic itself!!
Without telnetd running, there's no response (as expected). So clearly telnetd is running, listening to the port, responding to it, but just refusing to allow a login.
Any suggestions as to what's wrong? Is interactive login disabled or something?
BTW, this works wirelessly too -- don't need adb at all. Just get a shell prompt on the Epic and start telnetd. Then, you can use PuTTY (or any telnet client) to connect via wifi to <EpicIP>:23.
I tried this too, connected, but have the same instant disconnect problem.
I know this thread is old, but the solution might help someone. You need to start telnetd like this so it will invoke the shell when you telnet in:
I had to adb shell into the phone then run:
# busybox telnetd -l /system/bin/ash
As I'm new here I cannot post links, but if you google for 'Better cmd.exe terminal and ANSI color codes support' you will find another option then port forwarding etc.
Essentially using an alternative cmd.exe (Console) together with an ANSI helper program (ANSICON).
Did the trick for me at least.
Cheers,
Remco
http://forum.xda-developers.com/showthread.php?t=803223
ATTENTION (update on 2018-04-09): The procedures described in this thread are only working if you own an internet account with a public IPv4 address or dual stack i.e. both, public IPv4 and public IPv6 addresses. For account with only a public IPv6 address, it won't work. Please also refer to post #16.
Although I'd already read quite a lot about commercial VPN providers, reading of this article "VPN Leaks Found on 3 Major VPNs out of … 3 that We Tested" clearly established my decision to go for my own private VPN.
Thanks to Mike Kuketz who's running an excellent German blog regarding information technology security, I was able to study these two articles (https://www.kuketz-blog.de/pi-hole-schwarzes-loch-fuer-werbung-raspberry-pi-teil1/, https://www.kuketz-blog.de/pivpn-raspberry-pi-mit-openvpn-raspberry-pi-teil3/) about Pi-hole and PiVPN on/via a Raspberry Pi and immediately decided to purchase a Raspberry Pi 3 Model B+ (including an official case and charger) from an authorised Raspberry Pi dealer. Remark for German speaking XDA users: Mike also runs a very interesting forum in conjunction with his blog.
I'd be glad if this thread raises or raised your interest in a Raspberry Pi with Pi-hole and PiVPN. We are fascinated by their capabilities and glad to be able to utilise our own private VPN. If you also decide to go for it I hope that this tutorial facilitates setup and configuration. However, always be aware and remember that different scenario exist why use of a VPN might be reasonable. To anonymously browse the web via a VPN-provider certainly doesn't belong to that. The desire for anonymity and privacy in the world wide web is a reasonable wish of many users that can unfortunately hardly be implemented or only by extremely high efforts. You do not achieve anonymity while browsing the web, only because your network traffic is tunneled via a VPN-provider. This is only a promotional promise belonging into the category of modern fairy tales of the internet. However, by use of a (private) VPN you certainly enhance your privacy due to the encryption of your data traffic in this case between the Android device and the Raspberry Pi / PiVPN.
Intent of this thread is to share my experiences and procedure during the setup of the Raspberry Pi, Pi-hole and PiVPN. As client (or you might call it the companion of PiVPN) on our Android devices, I use OpenVPN for Android by Arne Schwabe. I downloaded it from F-Droid; however, it's also available via the Google Play Store. Possibly interesting to a few Android users might be that it does not require root. The whole setup is positively working on our Android Nougat ROM but I don't have any experiences with Android Oreo.
Additionally I want to clearly emphasise that I personally used Mike's two above linked articles written in German i.e. my thread is more or less only a translation of Mike's instruction into English. Therefore, I must clearly state that all credits go to Mike Kuketz.
Generally, in this thread I don't intend to discuss the reasons that induced my decision to establish my own private VPN or to create my own Network-wide ad blocking. Already brief searches of the web are providing multiple hits in this context but it's anyway a very private decision.
Additionally, I'm only focusing on our router, an AVM Fritz!Box 7390, our Android devices (Samsung Galaxy S3 LTE - i9305, all with RR-N-v5.8.5-final, Magisk v16.0, Xposed, XPrivacyLua and GApps-free thanks to microG), and Windows 10 Pro on a notebook (just started to familarise myself with Linux Mint i.e. all work in regard to this thread was conducted under Windows). I'm convinced that all interested readers of this thread are capable to translate/transfer the basic ideas to other routers, devices or Linux, iOS etc.
Content:
Post #2: Initial Installation of the Raspberry Pi
Post #3: The Pi-hole
Post #4: PiVPN
Post #5: PiVPN in Combination with the Pi-Hole
Post #6: OpenVPN for Android
Post #7: Dynamic DNS
Post #8: Customisation of the NTP-Server
Post #9: Unbound / Recursive DNS server
Remark:
In the attached screenshots, IP-addresses are blacked out for privacy reasons.
Please advise if something is not clear, incorrect or incomplete.
Off topic comments are allowed as long they are generally related to the overall topic, are in the general interest of the followers of this thread and add value to the thread. Having fun is always welcomed here. The ultimate decision rests with me as the OP!
Initial Installation of the Raspberry Pi
Updated on 2019-03-17!
********************
Initial Installation of the Raspberry Pi
As already said I'd ordered a Raspberry Pi 3 Model B+ including the official housing and AC charger. Most likely the whole setup is going to work with other Raspberry Pi models but please note that Jacob Salmela, the developer of Pi-hole, recommends a system of 512 MB RAM. Brief remark, you're unable to place the Raspberry Pi in its housing with an inserted microSD card.
Talking about microSD cards, I use a 32 GB, class 10 card to host the Raspberry Pi's OS and the Pi-hole/PiVPN. I'm convinced that a 16 GB card is also suitable, even a 8 GB one might be sufficient. I personally didn't require a keyboard or screen for the Raspberry Pi as I connect to it via a Secure Shell (SSH).
I decided to use RASPBIAN, the official OS of the Raspberry Pi Foundation; however, there're other OS' available, just search the web. I'm using Raspbian Stretch Lite, which fully meets my requirements, and downloaded it here as a zip-file. Unzipped the file and inserted the microSD into my notebook. There are multiple ways described to flash the OS image to the SD but I decided to use the way via Win32DiskImager. The Win32DiskImager utility is available via its Sourceforge Project page as an installer file. I just exactly followed the instructions as provided on the last linked Raspberry Pi page.
After the image had been flashed to the SD I had to create a simple file called "ssh" in the /boot partition in order to be later on able to access the Raspberry Pi via SSH. As a Windows user, first I'd to install Ext2Fsd driver to be able to access the system partition. The microSD was now prepared and ready to use.
The Raspberry Pi was already sleeping in its housing, I inserted the microSD into the Pi, connected the Pi by a regular network cable to LAN3 of my Fritz!Box (LAN1 is used for the connection to my Genexis Hybrid Live! Titanium-54 running in bridge mode as fiber modem, the internet radio is connected to LAN4) and finally connected the Pi to power.
For the following steps, please refer to the attached screenshots (I apologise for not havin changed Windows system language to English). Next step was to access the admin panel of our Fritz!Box. Its DHCP server is enabled; however, I don't allow the DHCP server to use the complete spectrum of IP addresses ("Home network => Home network overview => Network settings => IPv4-addresses"). On "Home network => Home network overview" I selected the details of the raspberrypi. Here, I assigned an IP to raspberrypi that is outside of the DHCP IP-range and ticked the always assign the same IP. Just for completeness, even before I installed the Raspberry Pi the DNS-servers were set to 85.214.20.141 (i.e. Digital Courage) and 213.73.91.35 (i.e. Chaos Computer Club) in the Fritz!Box ("Internet => Access credentials => DNS server"). On this German page you find other uncencored and free DNS server without tracking.
Knowing the IP-address of the Raspberry Pi, I now connected to the Pi via SSH by use of PuTTY that I downloaded from here and installed it. After start of PuTTY and entering of the Pi's IP, a terminal opens.
The default user credentials are:
- User: pi
- Password: raspberry
Now, I accessed the Pi's admin terminal, and first changed the default password by:
Code:
passwd
Changed slightly the Pi's configuration:
Code:
sudo raspi-config
Code:
Advanced Options → Expand Filesystem
Localisation Options → Change Timezone → Europe → Berlin
Finish, Reboot
My last step in the setup of the Raspberry Pi was to update the package by:
Code:
sudo apt-get update
sudo apt-get upgrade
sudo reboot
Final remark: I keep the Raspberry Pi's WiFi disabled as I don't require it.
The Pi-hole
Updated on 2019-03-18!
********************
The Pi-hole
The Pi-hole has been developed by Jacob Salmela since 2015. Pi-hole is based on dnsmasq and the webserver Lighttpd. The complete source code is available at GitHub. But what makes Pi-hole actually so special? It's a solution to block advertisement and trackers already within the network i.e. Pi-hole is theoretically able to blocks ads for all devices connected to the network. I guess this initially sounds adventurously but it proves to work in our home network.
If interested in the technical background please refer to the linked websites.
For the installation of Pi-hole on the Raspberry Pi, I connected to the Pi via SSH and opened a terminal. For a full automatic installation of Pi-hole I used the following command line:
Code:
curl -sSL https://install.pi-hole.net | bash
Attention: Please acknowledge the following statement posted on the Pi-hole webpage:
Our code is completely open, but piping to bash can be dangerous. For a safer install, review the code and then run the installer locally.
Click to expand...
Click to collapse
After completion of the installation of all packages and dependencies, the configurator opened. My personal selection is as follows:
Select Upstream DNS Provider
Custom: 85.214.20.141, 213.73.91.35 [Remark: DNS servers as already mentioned in post #2.]
Select Protocols
IPv4: Check
IPv6: Uncheck (Remark: None of our devices uses IPv6.)
Do you want to use your current network settings as a static address?
IP address: xxx.xxx.xxx.xxx (Remark: The fixed IP-addess of the Raspberry Pi.)
Gateway: xxx.xxx.xxx.1 (Remark: The IP of my router i.e. the Fritz!Box.)
Do you want to log queries?
On: Check
After the configurator's queries were completed it provided me with the address of graphical web-interface (http://pi.hole/admin or http://"IP-address of the Pi"/admin; screenshot available in the OP) and the login password for Pi-hole.
Remark: As soon as practicable I changed the initial password to my own one by following command line:
Code:
sudo pihole -a -p
In order that ads and trackers are blocked by the Pi-hole, it's necessary to point the Pi as the DNS-server to all devices. As usually, different ways and approaches exist to do so. Below I only describe the one I used.
Please refer to the attached screenshot that I already used in post #2, too. I circled the field where I inserted the IP-address of the Pi as the local DNS server.
Remark: With some routers it's possible to simply assign the IP-address of the Raspberry Pi as the new DNS-server. Advantage: Nothing is changing for the clients; they simply send a DNS-request to the router that forwards it to the Pi-hole in turn. However, this feature is not available for all Fritz!Boxes due to their integrated "DNS Rebind Protection".
Just for completeness a few useful Pi-hole commands:
pihole -h: Help that shows a list of all available commands
pihole -up: Initiates an update of the Pi-hole software
pihole -r: Relauch of the configurator e.g. to conduct changes to the DNS
pihole -g: Initiates an update of the blocklists
Pi-hole automatically updates the ad sources once a week on Sunday at a random time in the early morning. If required this "cron-job" can be changed via
Code:
sudo nano /etc/cron.d/pihole
respectively
Code:
sudoedit /etc/cron.d/pihole
Since Pi-hole version 3.x, it's no longer required to add/delete/amend blocklists via a terminal but can easily be accomplished via the Admin-web-interface.
Now some initial changes to the pi-hole settings via the Admin GUI:
Settings → DNS → DNSSEC: Enabled.
Settings → Blocklists: Set to you're own desire; I've got all default lists enabled. Personally I added the Non-crossed-list to the blocklists. Just copy and paste all lists into the text field, followed by a click onto "Save and Update".
In the dashboard, about 1M blocked domains should be indicated.
Final remark: Personally, I recognise the Pi-hole as my first line of defense, and I continue to use addons in my browser like uBlock Origin to defeat the rest.
PiVPN
Updated on 2019-03-18
*******************
PiVPN
The project PiVPN owns a webpage and additionally a Github-page, where it's source could can be examined. Basically, PiVPN is nothing else than a collection of shell scripts that facilitates installation and configuration of OpenVPN extremely.
I guess it's obvious that VPN only makes sense if the Android device is always able to reach the end of the tunnel and to connect to the Raspberry Pi. You are certainly aware that a lot of or most Internet Service Provider (ISP) assign dymnamic IPs to an Internet account - at least mine does i.e. my ISP regularly or occasionally changes the IP-address of my account. In turn, this means we need to ensure that the Android device "finds" the Raspberry Pi independent of its IP address. Two simple steps are required to achieve this and ought to be conducted prior to the installation of PiVPN on the Raspberry Pi:
Assign a static IP to the Raspberry Pi on the router as described in post #2.
Find and use a DynDNS-provider who converts the dynamic, public IP-address assigned by the ISP into a permanent domain name as described in post #7.
Remark: Ideally, use of the subnets 192.168.0.x/24 oder 192.168.1.x/24 should be avoided as they are very commonly in use, and routing conflicts might arise if trying to connect from the outside. In this context, please acknowledge a note taken from the OpenVPN-log:
NOTE: Your local LAN uses the extremely common subnet address 192.168.0.x or 192.168.1.x. Be aware that this might create routing conflicts if you connect to the VPN server from public locations such as internet cafes that use the same subnet.
Click to expand...
Click to collapse
After I managed these prerequisites, I commenced installation of PiVPN that is as easily conducted by a single command line as it had been for the Pi-hole (the respective attention note I made in post #3 also applies here):
Code:
curl -L https://raw.githubusercontent.com/pivpn/pivpn/master/auto_install/install.sh | bash
At first, the script updates the APT-package sources followed by the upgrade of the packages and subsequently installs OpenVPN.
During the installation I was able to customise my configuration. Attached are a few screenshots that I explain in sequence below:
As already stated the IP-address of PiVPN respectively the Raspberry Pi ought to be static on the router. The gateway address is usually the internal IP address of the router.
Usually, I'm not one for automated updates or upgrades as I rather maintain control and prefer to be able to immediately intervene in case of issues. However, I decide to make an exception for PiVPN as in this case activation of validation and installation of security updates seems to be very reasonable especially if the solution is meant to be as "fire (i.e. install) and forget"; i.e. install once and gotta rarely care. Don't interpret rarely as never; the automated security updates merely lighten my workload.
As protocol I chose UDP and left the standard port 1194 unchanged. At this point, I don't intend to start a discussion about the pro's or con's of OpenVPN via UDP or TCP, just briefly: UDP is faster and TCP more reliable. Please allow me to quote the OpenVPN mainpage:
OpenVPN is designed to operate optimally over UDP, but TCP capability is provided for situations where UDP cannot be used. In comparison with UDP, TCP will usually be somewhat less efficient and less robust when used over unreliable or congested networks.
Click to expand...
Click to collapse
Since OpenVPN v2.4, authentification and key exchange is possible via elliptic curves. PiVPN optionally generates either a 256-, 384-, or 521-bit-ECDSA-key pair, containing the public and private keys. 256-bit is the default setting, which is ok as it matches a 3072-bit.
The key generation on a Raspberry Pi 3 only takes a few seconds.
The striked-out lines are only valid for clients that doesn't support OpenVPB v2.4+:
For asymmetric keys, general wisdom is that 1024-bit keys are no longer sufficient to protect against well-equipped adversaries. Use of 2048-bit is a good minimum. It is wise to ensure all keys across your active PKI (including the CA root keypair) are using at least 2048-bit keys.
Up to 4096-bit is accepted by nearly all RSA systems (including OpenVPN,) but use of keys this large will dramatically increase generation time, TLS handshake delays, and CPU usage for TLS operations; the benefit beyond 2048-bit keys is small enough not to be of great use at the current time. It is often a larger benefit to consider lower validity times than more bits past 2048, but that is for you to decide.
Click to expand...
Click to collapse
Due to the already a few times mentioned "issue" with the dynamic public IP-address issued by the ISP, I ticked "Use a public DNS".
In this window I entered my domain name as mentioned in post #7 regarding dynamic DNS.
I selected custom to use the DNS servers of my choice.
Here, I entered "my" DNS servers as already explained in post #2.
This completed installation and configuration of PiVPN. Now, I had to create profiles that in turn need to be "installed" on my clients. Personally, I decided to use a distinct profile for each client . To create a profile for an Android device the follwoing command lines apply:
Code:
pivpn add
Code:
Enter a Name for the Client: MyClientName
Enter the password for the client: MyPassword
Subsequently the profile was generated with all necessary information (certificate, encryption details, etc.) and saved at /home/pi/ovpns.
I downloaded and installed FileZilla on my Windows notebook, connected via FileZilla to the Raspberry Pi and copied the file "MyClientName.ovpn" at /home/pi/ovpns onto my notebook. I transfered this file to my Android device and imported it into OpenVPN for Android; please refer to post #6 for more information in this respect.
That was it - now I was nearly able to connect my Android via my own private VPN with PiVPN respectively our Raspberry Pi; the only missing step was to open the router's/Fritz!Box's UDP port 1194 for the Raspberry Pi / PiVPN to allow data to pass from the outside.
The procedure is pretty simple and straight forward for a Fritz!Box (please refer to the last three screenshots). Open the admin web-interface of the Fritz!Box and select "Internet => Permissions => Port permissions => New port permission" (Remark: The English web-interface might probably read different than my translation but I'm convinced it's self-explaining). The IP must be the fixed IP assigned to the Raspberry Pi, I chose to name this permission "OpenVPN", selected UDP as the protocol and port "1194". And I didn't forget to tick the "Activate permission".
Last but not least, the following command line allowed me to check if my i9305 successfully connected to my PiVPN:
Code:
pivpn list
PiVPN in Combination with the Pi-Hole
Updated on 2019-06-15.
--------------------------------------------------------------------------------------------------------------------------------------------------
PiVPN in combination with the Pi-Hole
Please allow me to mention of another great advantage of having PiVPN together with Pi-hole on one and the same Raspberry Pi:
All of our mobile devices, which connect via OpenVPN with our home network, benefit from the Pi-hole i.e. no advertisement or trackers that follow us at every turn when connected to the web via mobile data or a WiFi network other than ours.
However, in order to achieve this I was require to slightly modify two configuration files on the Raspberry Pi as described below (please refer to the screenshots) - and ok, it's self-evident that I had to first install Pi-hole and PiVPN on the same Raspberry Pi before as described in this thread.
At first, I modified the OpenVPN server configuration by nano via the Raspberry Pi's console:
Code:
sudo nano /etc/openvpn/server.conf
The file opened and I looked for those two lines showing the IP-addresses of the DNS servers of my choice and as mentioned in the posts above:
Code:
push "dhcp-option DNS 85.214.20.141"
push "dhcp-option DNS 213.73.91.35"
I deleted one line and modified the other one to read:
Code:
push "dhcp-option DNS 10.8.0.1"
As DNS-server for all of our clients I've therefore defined the IP address of the VPN interface (tun0) (originally the local IP of the eth0 interface) of our Raspberry Pi, and hence forward all DNS-requests to the local DNS-server (dnsmasq) of the Pi-hole.
With its latest release Pi-hole changed the content of dnsmasq.conf located at /etc (for details refer to DNS Resolver in the Pi-hole documentation). dnsmasq.conf now simply points to a new folder named dnsmasq.d that is also located at /etc (refer to attached screenshot 1). This folder now contains the actual configuration files and is initially only populated with one file called 01-pihole.conf, which is the configuration file of Pi-hole's dnsmasq. 01-pihole.conf is used and modified by Pi-hole itself, and no custom modification should be made to it (refer to screenshot 2). However, additional configuration files in this folder will be executed in sequence by dnsmasq / FTLDNS.
This means I created a new file called 10-general.conf with the content:
Code:
cd /etc/dnsmasq.d
sudo touch 10-general.conf
sudo nano /etc/dnsmasq.d/10-general.conf
Insert line:
Code:
interface=tun0
This means we added a line with the VPN interface (tun0) that is listening on IP 10.8.0.1 by default.
Finally, I simply rebooted the Raspberry Pi.
OpenVPN for Android
OpenVPN for Android
As already stated in the OP that also contains a few screenshots, I only use OpenVPN for Android by Arne Schwabe on our Android devices. No experiences with other OpenVPN applications and most likely never will because Arne's app is easy to configure and perfectly running and performing as expected by me.
During the installation of PiVPN and as explained in post #4 I created profiles for each of our mobile Android devices. I transfered the respective profile file (name something like "MyClientName.ovpn") to the respective device. I installed "OpenVPN for Android", opened the application, granted permission to "storage" and just imported the before mentioned file. If I correctly remember the application questions the password I created during the creation of the profile. This password is always queried when starting on OpenVPN connection. That was all; I didn't modify anything in the settings of the application.
Please acknowledge these Security Considerations provided by Arne on his FAQ page:
"As OpenVPN is security sensitive a few notes about security are sensible. All data on the sdcard is inherently insecure. Every app can read it (for example this program requires no special sd card rights). The data of this application can only be read by the application itself. By using the import option for cacert/cert/key in the file dialog the data is stored in the VPN profile. The VPN profiles are only accessible by this application. (Do not forget to delete the copies on the sd card afterwards). Even though accessible only by this application the data is still unencrypted. By rooting the telephone or other exploits it may be possible to retrieve the data. Saved passwords are stored in plain text as well. For pkcs12 files it is highly recommended that you import them into the android keystore."
Click to expand...
Click to collapse
Dynamic DNS
EDIT (2018-06-15)
--------------------------------------------------------------------------------------------------------------------------------------------------
Dynamic DNS
You are certainly aware that a lot of or most Internet Service Provider (ISP) assign dymnamic IPs to an Internet account - at least mine does i.e. my ISP regularly or occasionally changes the IP-address of my account. You can easily retrieve the IP address currently assigned to your account by e.g. IP/DNS Detect (and which additionally offers a lot of other useful information about your current footprint in the web - or how well they are disguised by your browser addons).
However, with Dymnamic DNS it's possible to connect to my Fritz!Box respectively the Raspberry Pi despite the changing IPs by use of an unchanging domain name. In order to actually achieve this, a dynDNS-provider is required. I personally went with Two-DNS that offers an account with up to five free hosts and to choose from a wide collection of domains. Below I try to explain how I configured our Fritz!Box for the use of Two-DNS.
By default, the Fritz!Box already cooperates with a lot of dynDNS-providers but not with Two-DNS; however, via the option "user defined/customised" it's pretty easily achieved.
Create your Two-DNS account.
After you created the account, a "New Host" is created by Two-DNS.
You can choose any host name you want unless it's already in use. The dropdown box offers you different possibilities for the domain (part).
Just as an example; it's not my actual domain name (in blue the host name; in green the domain): "myfritzbox.my-wan.de". The Fritz!Box respectively the Raspberry Pi can later be called up by this domain name.
This completed the setup of the account/host at Two-DNS. Now we need to access the web-interface of the Fritz!Box.
Via "Internet => Permissions => Dynamic DNS" (I hope the settings translate this way to English but I'm convinced you're figuring it out) the following settings were assigned:
Dynamic DNS-Provider: User defined/customised
Update-URL: https://update.twodns.de/update?hostname=<domain>&ip=<ipaddr>
Domain name: your domain (e.g. myfritzbox.my-wan.de)
User name: the email address you registered with Two-DNS
Password: your Two-DNS password
Apply - and that's it. On "Internet => Online-Monitor" it should read (refer to screenshot):
DynDNS active, "your domain name", IPv4-status: successfully logged in.
Attention: Due to the limited number of IPv4-addresses, a lot of new internet account have been connected to the internet via Dual Stack Lite (DS-Lite). If this is the case for your internet account, above mentioned procedure is unusable. Please acknowledge following AVM post on their website in this respect.
Customisation of the NTP-Server
I've customised the NTP-server to synchronise the time with the German Physikalisch-Technische Bundesanstalt (PTB).
Code:
sudo nano /etc/systemd/timesyncd.conf
NTP=ptbtime1.ptb.de ptbtime2.ptb.de
Unbound / Recursive DNS server
Updated on 2019-03-18
******************
Unbound / Recursive DNS server
I decided to install Unbound in order to operate the Pi as my own (tiny) recursive DNS server.
Via the Pi-hole admin GUI, I disabled DNSSEC in Settings => DNS, as Unbound is handling that later on.
As we require for the very last step , the local root zone, a new version of Unbound than the one currently available via the default sources of Raspbian Stretch we need to play tricky via Apt-Pinning to allow to retrieve the software from the testing branch of Debian.
We install dirmngr and fetch a GPG key to verify the downloaded packages from the testing branch:
Code:
sudo aptitude install dirmngr
sudo apt-key adv --receive-keys 0x7638D0442B90D010
Now we edit the sources.list and add the link to the package.
Code:
sudo nano /etc/apt/sources.list
#Testing
deb http://ftp.de.debian.org/debian/ testing main non-free contrib
Now we give the testing branch a lower priority than stable:
Code:
sudo nano /etc/apt/preferences
Package: *
Pin: release a=stable
Pin-Priority: 600
Package: *
Pin: release a=testing
Pin-Priority: 400
Update of the database and installation of Unbound:
Code:
sudo aptitude update
sudo aptitude install unbound/testing
During the installation, Raspbian provides suggestions how to resolve the dependencies. First suggestion is to simply not install Unbound what we deny by "N". In the second suggestion, all current dependencies ought to be updated from the testing branch what we confirm by two times "Y". And we allow the services to be automatically re-started during the installation. Don't care about possible red error messages; we'll take care of that later.
During the installation you'll see following message:
Configuration file '/etc/lighttpd/lighttpd.conf'
==> Modified (by you or by a script) since installation.
==> Package distributor has shipped an updated version.
What would you like to do about it ? Your options are:
Y or I : install the package maintainer's version
N or O : keep your currently-installed version
D : show the differences between the versions
Z : start a shell to examine the situation
The default action is to keep your current version.
Click to expand...
Click to collapse
Please answer with "N" in order to keep the Lighttpd configuration that was installed by Pi-hole.
In order to avoid network issues per DHCP during a network re-start to add to following file:
Code:
sudo nano /etc/network/interfaces
Code:
[FONT=Verdana] auto lo[/FONT]
[FONT=Verdana]iface lo inet loopback[/FONT]
[FONT=Verdana]
[/FONT]
[FONT=Verdana]auto eth0[/FONT]
[FONT=Verdana]iface eth0 inet dhcp[/FONT]
Now we provide Unbound with a file containing name and address of the root server.
Code:
wget -O root.hints https://www.internic.net/domain/named.root sudo mv root.hints /var/lib/unbound/
We adapt the additional config-file for Unbound provided by Pi-hole:
Code:
sudo nano /etc/unbound/unbound.conf.d/pi-hole.conf
server: # If no logfile is specified, syslog is used # logfile: "/var/log/unbound/unbound.log" verbosity: 0 port: 5353 do-ip4: yes do-udp: yes do-tcp: yes # May be set to yes if you have IPv6 connectivity do-ip6: no # Use this only when you downloaded the list of primary root servers! root-hints: "/var/lib/unbound/root.hints" # Trust glue only if it is within the servers authority harden-glue: yes # Require DNSSEC data for trust-anchored zones, if such data is absent, the zone becomes BOGUS harden-dnssec-stripped: yes # Don't use Capitalization randomization as it known to cause DNSSEC issues sometimes # see https://discourse.pi-hole.net/t/unbound-stubby-or-dnscrypt-proxy/9378 for further details use-caps-for-id: no # Reduce EDNS reassembly buffer size. # Suggested by the unbound man page to reduce fragmentation reassembly problems edns-buffer-size: 1472 # TTL bounds for cache cache-min-ttl: 3600 cache-max-ttl: 86400 # Perform prefetching of close to expired message cache entries # This only applies to domains that have been frequently queried prefetch: yes # One thread should be sufficient, can be increased on beefy machines num-threads: 1 # Ensure kernel buffer is large enough to not lose messages in traffic spikes so-rcvbuf: 1m # Ensure privacy of local IP ranges private-address: 192.168.0.0/16 private-address: 169.254.0.0/16 private-address: 172.16.0.0/12 private-address: 10.0.0.0/8 private-address: fd00::/8 private-address: fe80::/10
Re-start Unbound and let's test functionality, commencing with a simple DNS request followed by DNSSEC:
Code:
sudo systemctl restart unbound
dig kuketz-blog.de @127.0.0.1 -p 5353 dig sigfail.verteiltesysteme.net @127.0.0.1 -p 5353 dig sigok.verteiltesysteme.net @127.0.0.1 -p 5353
The second request should provide "status: SERVFAIL" while the last one a "status: NOERROR".
Now go back into the Pi-Hole admin GUI, and under Settings => DNS delele the entry in "Custom 2" and untick it.
For "Custom 1" modify the entry to:
127.0.0.1#5353
Click to expand...
Click to collapse
Pi-hole lighttpd Workaround
The upgrade of Debian Buster Packages (Testing) of Unbound (v. 1.9.0-2) also upgrades the Lighttpd-Webserver-Package from version 1.4.45-1 (now, stable) -> 1.4.53-3 (testing). Currently, Pi-hole isn't yet compatible with the new Lighttpd-Syntax. After an upgrade to the new version, the Lighttpd webserver doesn#t start and the Pi-hole web interface can't be reached. In order to solve this issue, here's the follwoing work-around:
Code:
sudo nano /etc/lighttpd/lighttpd.conf
Comment the following line
Code:
#include_shell "/usr/share/lighttpd/create-mime.assign.pl"
and insert this line:
Code:
include_shell "/usr/share/lighttpd/create-mime.conf.pl"
Start or re-start the lighttpd service:
Code:
service lighttpd start
service lighttpd restart
Some users, including me, complained about starting issues of the lighttpd webserver, therefore I also commented the following line:
Code:
#include_shell "cat external.conf 2>/dev/null"
In order to spare us the first DNS request by Unbound to the DNS root server, we provide Unbound with the respective configuration. It must occasionally be updated, as the DNS root server themselves sometimes receive changes e.g. their IP addresses. As I don't want to update everything manually, I created to things:
a mechanism that automatically notifies my about updates
and a script that eases the replacement by new configuration files.
We create a dynamic Message of the Day (MOTD). This is supposed to be a notification that always appears whenever we log into the Pi via SSH. We delete the static MOTD in order to only have our dynamic one be displayed.
Code:
sudo rm /etc/motd
We now edit a file in that folder that is analysed for the creation of MOTD's. This will ensure that we're always informed about the currentness of the Hyperlocal configuration when we log into the Pi via SSH.
Code:
sudo nano /etc/update-motd.d/20-info
The codes of this file will be executed and the resulting output transferred to the MOTD. Add the following lines but replace "NAME" with your Pi user name.
Code:
#!/bin/bash
echo
echo -e "\e[1mUptime:\e[m $(uptime)"
echo -e "\e[1mDate:\e[m $(date)"
echo -e "\e[1mHyperlocal conf:\e[m $(cat /home/NAME/.unbound/update.txt)"
echo
In the folder for user scripts we now create the new script that will provide the update notifications:
Code:
sudo nano /usr/local/bin/autoupdatelocalroot
Code:
#!/bin/bash ## VARIABLES ## DIR=$HOME/.unbound hints=/var/lib/unbound/root.hints conf=/etc/unbound/unbound.conf.d/localroot.conf infile=${DIR}/root.hints outfile=${DIR}/localroot.conf update=${DIR}/update.txt ## SCRIPT ## # check for existence of update.txt file and .unbound directory if [[ ! -d $HOME/.unbound ]]; then mkdir ${DIR} fi if [[ ! -e $HOME/.unbound/update.txt ]]; then echo "up to date" > ${update} fi # get the file with the root servers and save as "root.hints" wget --timeout=30 -O ${infile} https://www.internic.net/domain/named.root # extract name and IP addresses (A + AAAA) of root servers and nicely put them into the file for unbound awk '\ BEGIN\ { print "auth-zone:\n\tname: \".\"" } { if($0 ~ /[ ]NS[ ]/) { print "\t# "$NF } if($0 ~ /[ ]A[ ]/) { print "\tmaster: "$NF } if($0 ~ /[ ]AAAA[ ]/) { print "\tmaster: "$NF } } END\ { print "\tfallback-enabled: yes\n\tfor-downstream: no\n\tfor-upstream: yes\n\tzonefile: \"root.zone\"\n" }\ ' ${infile} > ${outfile} #update the motd update notification if neither outfile nor diff file empty if [[ -e ${outfile} && "$(diff -Niw ${conf} ${outfile})" != "" ]] || [[ -e ${infile} && "$(diff -Niw ${hints} ${infile})" != "" ]]; then echo "Update available – please run: sudo updateunboundconf" > ${update} else echo "up to date" > ${update} fi #print update status cat ${update} echo
The script will be executable for all users and be added to Crontab for regulat execution. When I was asked, which editor to use while editing Crontab, I stayed with nano.
Code:
sudo chmod 755 /usr/local/bin/autoupdatelocalroot
crontab -e
In order to e.g. execute the script on Sundays at 04:20 we add the following line:
Code:
20 4 * * 0 /usr/local/bin/autoupdatelocalroot
Now we create the second script that will ease the update of the configuration files. It won't be executed automatically but only after a manual launch for a simple reason: I won't to control this and be personally present in case of any unforeseen event.
Code:
sudo nano /usr/local/sbin/updateunboundconf
Code:
#!/bin/bash ## VARIABLES ## DIR=/home/$(logname)/.unbound hints=/var/lib/unbound/root.hints conf=/etc/unbound/unbound.conf.d/localroot.conf infile=${DIR}/root.hints outfile=${DIR}/localroot.conf update=${DIR}/update.txt PLSUPDATE="Please run 'autoupdatelocalroot' first." NOTHING="Update skipped, nothing done." ## SCRIPT ## #update root.hints file if [[ -e ${infile} ]] && [[ "$(diff -Niw ${hints} ${infile})" != "" ]]; then input=r echo "Install new root.hints file for Unbound (overwrites old file)?" echo "Yes / No / Re-Read differences?" while [[ "$input" =~ [rR] ]]; do diff -Niw ${hints} ${infile} | less read -e -p " [Default = no] (y/n/r): " input done if [[ "$input" =~ [yY] ]]; then mv -fv ${infile} ${hints} chown unbound:unbound ${hints} chmod 644 ${hints} yes1=TRUE else echo echo $NOTHING echo fi else if [[ ! -e ${infile} ]]; then echo echo $PLSUPDATE echo exit 1 else yes1=TRUE fi fi #update localroot.conf file if [[ -e ${outfile} ]] && [[ "$(diff -Niw ${conf} ${outfile})" != "" ]]; then input=r echo "Install new localroot.conf file for Unbound (overwrites old file)?" echo "Yes / No / Re-Read differences?" while [[ "$input" =~ [rR] ]]; do diff -Niw ${conf} ${outfile} | less read -e -p " [Default = no] (y/n/r): " input done if [[ "$input" =~ [yY] ]]; then mv -fv ${outfile} ${conf} yes2=TRUE else echo echo $NOTHING echo fi else if [[ ! -e ${outfile} ]]; then echo echo $PLSUPDATE echo exit 1 else yes2=TRUE fi fi #update motd update notification if [[ "$yes1" == TRUE ]] && [[ "$yes2" == TRUE ]]; then echo "up to date" > ${update} echo echo "Unbound's local root config is up to date!" echo else echo echo "Entire or partial Update still pending." echo fi
This script also gets the permissions to be executable; however, only for root users (in case you use multiple users).
Code:
sudo chmod 744 /usr/local/sbin/updateunboundconf
Let's perform a test run and hereby simultaneously create the effective configuration files.
Code:
sudo autoupdatelocalroot
For the second script we're using the toll diff that illustrates the diffences betwenn the files. The symbol "<" at the beginning of a line shows that this line is removed in the second file (i.e. the new, updated configuration), while ">" means this line will be added. To visit all differences, navigate with the arrow keys and terminate with the key "q".
For final installation of the new files confirm with "Y".
Code:
sudo updateunboundconf
All credits go to Mike Kuketz and Max Tschaeggaer for their German speaking tutorial.
reserved #9
reserved #10
Congratulations on putting together this great and well-written guide :good:
I was thinking about something similar to Pi-hole, but never thought about the possibility to combine ad-blocking with my own VPN. Thank you for bringing this to my attention, I surely have some things left to read on the security blog by Mike Kuketz.
Portgas D. Ace said:
Congratulations on putting together this great and well-written guide :good:
I was thinking about something similar to Pi-hole, but never thought about the possibility to combine ad-blocking with my own VPN. Thank you for bringing this to my attention, I surely have some things left to read on the security blog by Mike Kuketz.
Click to expand...
Click to collapse
Congratulations to your 4,000th post.
I was in contact with Mike and he granted me permission to more or less translate his tutorials and to post on XDA. Already a long time ago I realised that's really worth to monitor his blog closely.
Nice work and as always precise in your instructions :good:
Sent from my Pixel 2 XL using XDA Labs
Meanwhile, I also have a fixed public IPv4-address. I found a company that corporates with my ISP, and they provided my with the fixed IPv4 for a small monthly fee. All procedures described above (especially in post #4 and post #7) remain the same with one exception: I did not enable respectively setup DynDNS in the Fritzbox.
Important also, you need to find a DynDNS-provider that allows you to manually insert an IPv4-address into your DynDNS-account; not all providers do, e.g. the in post #7 mentioned Two-DNS doesn't. These providers simply take the IPv4-address that they read, and that one is a non-public dynamic IPv4 address. I'm now with ddnss.de, Here, I was allowed to manually override and save the read nun-public IPv4 by my new fixed IPv4-address.
The reason why you must not enable DynDNS in the Fritzbox is pretty easy: If it is enabled it will initiate updates of your DynDNS and hence overwrite the fixed public IPv4-address by a dynamic non-public IPv4 through that the Pi can't be accessed.
PiVPN is fantastically running, and I've OpenVPN on all of our devices now, which have access to the internet. Now, and doesn't matter where, or if mobile data, an unsecure public WiFi network or a secure WiFi network other than ours is used, I'm able to initiate my own private secure VPN tunnel to my router respectively my RaspberryPi.
Having this working now, I'm going to stick with my current ISP-provider.
EDIT (2018-06-24): Just for completeness - while on mobile network with the Android, the VPN with my RaspberryPi is established in about 3 seconds on WiFi and 5 seconds on mobile data (even if only on 2G). To establish the VPN between the PC and the Pi it takes 7 seconds (see attached log). Additionally, use of the Pi as DNS-server works seamlessly.
Hi! I followed your guide and everything is working properly. I have just one issue. I would like to route only DNS via VPN. I tried doing what this article in Pi-hole documentation suggested. I couldn't find
HTML:
push "redirect-gateway def1 bypass-dhcp"
so I commented out
HTML:
push "redirect-gateway def1"
But doing this stops the internet connection on the client device though the openvpn profile is succesfully connected.
Any suggestions on how I can route only DNS via VPN? Any help would be much appreciated. Thanks
Ex-Hunter said:
Hi! I followed your guide and everything is working properly. I have just one issue. I would like to route only DNS via VPN. I tried doing what this article in Pi-hole documentation suggested. I couldn't find
HTML:
push "redirect-gateway def1 bypass-dhcp"
so I commented out
HTML:
push "redirect-gateway def1"
But doing this stops the internet connection on the client device though the openvpn profile is succesfully connected.
Any suggestions on how I can route only DNS via VPN? Any help would be much appreciated. Thanks
Click to expand...
Click to collapse
I couldn't find the line mentioned in the linked turorial either; my server.conf file also only contains
Code:
push "redirect-gateway def1"
, a line that is not commented out.
I apologise I've no clue at all. You certainly have your reasons to go the way you describe, and I'm not questioning it. Before I used OpenVPN with my Pi, I used the DNS changer I mentioned in about the middle of this post. I can confirm this application is working even with mobile data by just establishing a VPN in order to only use DNS servers of your desire.
In order to keep this thread up-to-date, I like to share my latest and new experiences. Till last Friday and as I already mentioned within this thread my internet connection was via fibre-optics. My data plan with the fibre-optics ISP was 100 MBit/s down- and upload, and as this ISP only offers public dynamic IPv6 addresses, I additionally had to book a public static IPv4 address from another commercial provider. Overall, my monthly charges for this setup were about 55 €.
Since last Friday, my new internet connection via VDSL by a different ISP is online. The data plan I ordered is for 50 MBit/s download and 10 MBit/s upload, which I assumed to fulfill my requirements. If I realise this to be insufficient I can upgrade to 100/40 anytime but after only three days I already doubt I have to. This ISP offers public dynamic IPv4 addresses i.e. I don't require the additional contract for a static IPv4 anymore (and it has already been cancelled). For this data plan, my new ISP charges me about 35€ a month i.e. I'm now saving 20€ per month.
With this change of ISP (and type of connection) I had to perform three changes in the settings of my FritzBox: Enter the new credentials for the different internet connection, enter the new credentials for VoIP, and re-enable the dynDNS that I did setup when I initially established the Pi in our home network.
Overall: My own private secure VPN between OpenVPN for Android and the PiVPN is continuing to work flawlessly and perfectly! DynDNS is always immediately updated when the public IPv4 address changes. I'm still extremely satified with the complete setup described in this thread.
Can you write on how to forward only DNS requests to the VPN via OpenVPN? I couldn't get it to work. Also, forwarding port 53 and using the pi as a public DNS is not recommended.
MikeTheGamer said:
Can you write on how to forward only DNS requests to the VPN via OpenVPN? I couldn't get it to work. Also, forwarding port 53 and using the pi as a public DNS is not recommended.
Click to expand...
Click to collapse
A few month ago, @Ex-Hunter asked a very similar question here. I'm unable to answer you're question. I've achieved my primary goal with my setup to be able to connect to my PiVPN via my own secure VPN if I'm connected to a network other than our home one. Additionally, I wanted to use the Pi-hole in our home network. All is working great! And I absolutely trust Mike Kuketz whom I mentioned in the OP and whose instructions I followed.