Related
The Vulnerability
In recent updates to some of its devices, HTC introduces a suite of logging tools that collected information. Lots of information. LOTS. Whatever the reason was, whether for better understanding problems on users' devices, easier remote analysis, corporate evilness - it doesn't matter. If you, as a company, plant these information collectors on a device, you better be DAMN sure the information they collect is secured and only available to privileged services or the user, after opting in.
That is not the case. What Trevor found is only the tip of the iceberg - we are all still digging deeper - but currently any app on affected devices that requests a single android.permission.INTERNET (which is normal for any app that connects to the web or shows ads) can get its hands on:
the list of user accounts, including email addresses and sync status for each
last known network and GPS locations and a limited previous history of locations
phone numbers from the phone log
SMS data, including phone numbers and encoded text (not sure yet if it's possible to decode it, but very likely)
system logs (both kernel/dmesg and app/logcat), which includes everything your running apps do and is likely to include email addresses, phone numbers, and other private info
Normally, applications get access to only what is allowed by the permissions they request, so when you install a simple, innocent-looking new game from the Market that only asks for the INTERNET permission (to submit scores online, for example), you don't expect it to read your phone log or list of emails.
But that's not all. After looking at the huge amount of data (the log file was 3.5MB on my EVO 3D) that is vulnerable to apps exploiting this vulnerability all day, I found the following is also exposed (granted, some of which may be already available to any app via the Android APIs):
active notifications in the notification bar, including notification text
build number, bootloader version, radio version, kernel version
network info, including IP addresses
full memory info
CPU info
file system info and free space on each partition
running processes
current snapshot/stacktrace of not only every running process but every running thread
list of installed apps, including permissions used, user ids, versions, and more
system properties/variables
currently active broadcast listeners and history of past broadcasts received
currently active content providers
battery info and status, including charging/wake lock history
and more
Let me put it another way. By using only the INTERNET permission, any app can also gain at least the following:
ACCESS_COARSE_LOCATION Allows an application to access coarse (e.g., Cell-ID, WiFi) location
ACCESS_FINE_LOCATION Allows an application to access fine (e.g., GPS) location
ACCESS_LOCATION_EXTRA_COMMANDS Allows an application to access extra location provider commands
ACCESS_WIFI_STATE Allows applications to access information about Wi-Fi networks
BATTERY_STATS Allows an application to collect battery statistics
DUMP Allows an application to retrieve state dump information from system services.
GET_ACCOUNTS Allows access to the list of accounts in the Accounts Service
GET_PACKAGE_SIZE Allows an application to find out the space used by any package.
GET_TASKS Allows an application to get information about the currently or recently running tasks: a thumbnail representation of the tasks, what activities are running in it, etc.
READ_LOGS Allows an application to read the low-level system log files.
READ_SYNC_SETTINGS Allows applications to read the sync settings
READ_SYNC_STATS Allows applications to read the sync stats
Theoretically, it may be possible to clone a device using only a small subset of the information leaked here.
I'd like to reiterate that the only reason the data is leaking left and right is because HTC set their snooping environment up this way. It's like leaving your keys under the mat and expecting nobody who finds them to unlock the door. For a more technical explanation, see the section below.
Additionally, and the implications of this could end up being insignificant, yet still very suspicious, HTC also decided to add an app called androidvncserver.apk to their Android OS installations. If you're not familiar with the definition of VNC, it is basically a remote access server. On the EVO 3D, it was present from the start and updated in the latest OTA. The app doesn't get started by default, but who knows what and who can trigger it and potentially get access to your phone remotely? I'm sure we'll know soon enough - HTC, care to tell us what it's doing here?
Technical Details
In addition to Carrier IQ (CIQ) that was planted by HTC/Sprint and prompted all kinds of questions a while ago, HTC also included another app called HtcLoggers.apk. This app is capable of collecting all kinds of data, as I mentioned above, and then... provide it to anyone who asks for it by opening a local port. Yup, not just HTC, but anyone who connects to it, which happens to be any app with the INTERNET permission. Ironically, because a given app has the INTERNET permission, it can also send all the data off to a remote server, killing 2 birds with one stone permission.
In fact, HtcLogger has a whole interface which accepts a variety of commands (such as the handy :help: that shows all available commands). Oh yeah - and no login/password are required to access said interface.
Furthermore, it's worth noting that HtcLogger tries to use root to dump even more data, such as WiMax state, and may attempt to run something called htcserviced - at least this code is present in the source:
/system/xbin/su 0 /data/data/com.htc.loggers/bin/htcserviced
HtcLoggers is only one of the services that is collecting data, and we haven't even gotten to the bottom of what else it can do, let alone what the other services are capable of doing. But hey - I think you'll agree that this is already more than enough.
Patching The Vulnerability
... is not possible without either root or an update from HTC. If you do root, we recommend immediate removal of Htcloggers (you can find it at /system/app/HtcLoggers.apk).
Stay safe and don't download suspicious apps. Of course, even quality-looking apps can silently capture and send off this data, but the chance of that is lower.
Affected Phones
Note: Only stock Sense firmware is affected - if you're running an AOSP-based ROM like CyanogenMod, you are safe.
EVO 4G
EVO 3D
Thunderbolt
EVO Shift 4G? (thanks, pm)
MyTouch 4G Slide? (thanks, Michael)
the upcoming Vigor? (thanks, bjn714)
some Sensations? (thanks, Nick)
View 4G? (thanks, Pat)
the upcoming Kingdom? (thanks, Pat)
most likely others - we haven't verified them yet, but you can help us by downloading the proof of concept above and running the APK
HTC's Response
After finding the vulnerability, Trevor contacted HTC on September 24th and received no real response for five business days, after which he released this information to the public (as per RF full disclosure Policy). In my experience, lighting fire under someone's ass in public makes things move a whole lot faster, which is why responsible disclosure is a norm in the security industry. (This is where we come in.)
As far as we know, HTC is now looking into the issue, but no statement has been issued yet.
HTC, you got yourself into this mess, and it's now up to you to climb out of the hole as fast as possible, in your own interest.
The ball is in your court.
Credit
ANDROID POLICE
Huge thank you to Trevor Eckhart who found the vulnerability and Justin Case for working with us today digging deeper.
Hi there, I need help, someone is consistently hacking into my phone, htc evo 4g, they are penetration testers and pc savvy, currently I cant login to the phn for trying to do a factory reset. They kept intercepting me and now my password does not work. Who knows maybe they changed it on their side. I wrote down everything I saw. I was seeing all these process running for the same app. in my applications. My phone was getting hot, freezes but its people that live in my apt complex and at work. can you help?
zzm5 said:
Hi there, I need help, someone is consistently hacking into my phone, htc evo 4g, they are penetration testers and pc savvy, currently I cant login to the phn for trying to do a factory reset. They kept intercepting me and now my password does not work. Who knows maybe they changed it on their side. I wrote down everything I saw. I was seeing all these process running for the same app. in my applications. My phone was getting hot, freezes but its people that live in my apt complex and at work. can you help?
Click to expand...
Click to collapse
Is your device rooted?
I used root explorer and removed the HtcLoggers.apk and other than the forced close loop that removing it caused (requiring me to remove the battery), after rebooting all seems to be working fine.
EDIT: Actually I didn't just delete HtcLoggers.apk but moved it to a safe location on the SD Card in case there was a problem and it needed to be restored. I highly suggest you do this instead of just deleting it, or better yet, a nandroid backup.
there are a few good ROMS out there that have the ICQ loggers removed already.
Do we really need three threads on the front page about the same thing?
Version Alpha 0.6.0 is now available
I'm back! Not dead yet, I promise. This is actually a relatively small update in terms of user-facing features, with only one really big new thing - support for file uploading - but that's a lot bigger than it might sound. It's the first write support I've implemented in the server, and it also required some fairly massive updates to the HttpServer component (support for binary requests, for POST parameters, for MIME multipart parsing). These will be built upon in forthcoming versions to add support for things like registry editing, in-browser file viewing (possibly editing), and so on. There are also a large number of small fixes and improvements that I've made over the last two-weeks-shy-of-a-year, which should make the server faster, more robust, better able to support concurrent connections, and lighter on device resources. Finally, while the app still targets WP8.0 and should run on 8.0, it now is designed for 8.1 compatibility (especially the AllCapabilities version).
Previous update (0.5.6): This version is mostly bug fixes and UI changes. The biggest changes are: clearer display of weird registry data types, the server now consumes fewer threads (it used to spawn them with wild abandon) and does faster string compares, the app version is now shown on the phone, error pages are now better, if you launch the app without a WiFi IP address it'll offer to take you to the WiFi settings page, connections are no longer closed as soon as the app starts sending a response, and the server now defaults to using the Connection: keep-alive header, with a two-minute timeout. The last change, combined with the second-to-last, should hopefully both do away with the tendency to have the app fail to display a page. However, I shouldn't have *needed* to switch it to "keep-alive" - using "close" should have worked - but it still veeeery occasionally would kill the connection early. Agh. Anyhow, this is better in the meantime.
DevDB offers me a support / Q&A thread. Please use that thread to ask questions; don't PM me unless it needs to be kept private for some reason!
ISSUES ON WP8.1:
It *should* work to deploy the app with "Application Deployment", but if you have a problem try deploying with "Windows Phone Application Deployment 8.1" instead.
Problems have been reported in the past when the app is installed to the SD card. It's small, though; putting it on internal storage shouldn't be a problem.
RESOLVED The AllCapabilities version included a few capabilities that were present in 8.0 but removed in 8.1. Those capabilities have been removed; the AllCapabilities version now deploys and runs on capability-unlocked WP8.1 phones.
IN CASE OF OTHER ISSUES: Please provide a *detailed* error report - what phone and OS version you have, what hacks you've installed, what Webserver version you're running, what you do to get the error to occur, and exactly *what* occurs - and I'll fix it as soon as I can! There's a DevDB section for posting bug reports, and you can also use CodePlex if you want.
I finally implemented file upload! I'll work on getting more stuff like that (file delete, possibly file rename/move/copy, various registry edits), hopefully soon! I also hope to add support for different areas, like an "Applications" path, a "Processes" path, a "Services" path... eventually. Many of those are really hard without good privileges. I'm also looking at moving the server to a background process and making the app just a control UI for it, adding support for authentication and/or HTTPS, adding some stylesheets to the web UI, adding caching, and much more. I did finally implement Connection header support.
Once again, the XAP is published twice. One is a fairly standard XAP that any phone can sideload, and the second has many exotic capabilities to enable viewing of (and writing to) slightly more of the file system and registry. The standard XAP has had its list of capabilities expanded to pretty much all of them that can be used without interop-unlock. The high-capability variant requires not just interop-unlock, but the additional capability-unlock hack available in the interop-unlock thread. The AllCapabilities version now works with WP8.1; sorry for the long delay on that!
An item of note: the AllCapabilities version (or either version, on WP8.1) can open other drives in the file system. On phones with an SD card, it is mounted at D: and you can browse it as normal. Credit to @hjc4869 for this discovery!
DESCRIPTION: This is a simple webserver app which can enumerate those files that are in folders readable from the sandbox, can download and upload (access permitting) files, can browse the registry, and can display the contents of registry values of any type. It runs on WP8.x (not yet tested on W10M). It is a spiritual successor to the Functional Webserver / WebServer (Mango) projects from WP7. This version is still missing a lot of functionality as I decided to implement it from scratch, but it is advancing swiftly. Note that there's no access controls implemented; use it on a public network only at your own risk!
Instructions are simple: sideload the XAP, connect to WiFi (required), run the app (called "WebServer Native Access"), point a web browser (on a PC or phone that is also on that local network) to the URL that the app displays. You should get a basic index page. Click on a Filesystem or Registry link to begin browsing the phone. There's a textbox near the top of all filesystem pages, type in a path there (for example, "C:windows" with no quotes) and hit Enter or click Get Files. You'll see a list of the contents of that folder. Click on a file to download it or a directory to open it. There's also a box for uploading files, one at a time, to the current directory. Navigating the registry is similar, except you'll need to specify the registry hive and then the path from that hive (or no path, to access the root of the hive).
As of v0.6.0, uploading files is finally supported! Other modifications (editing files, creating, deleting, or changing registry keys or values) are currently not supported. They will be "soon" although my personal testing suggests that basically the whole registry, and most of the file system, is off-limits for writing unless you use restricted capabilities.
You might see an error code (error 5 is "ACCESS_DENIED", you'll see it a lot; I should replace it with an appropriate 403 or whatever). Or you might see a status 500 message because of an exception in the server. Or the server may just crash (hopefully not so often anymore...). I'm making it more resilient, but there are still bugs. Please report any previously-unreported issues you find, including how to reproduce them, and I'll fix them if possible.
Also feel free to request features or changes; I'll implement them if reasonably possible. The app is a mixture of C++ and C# code; I could probably have done it all in one or the other but wanted to have a C++ component in case I ran into something that wasn't available in C#, and although it probably would have saved some time, I decided that hacking up a web server in C++ was maybe not the best idea.
The source code is on Codeplex, at the following projects: https://wp8webserver.codeplex.com/ for the server and the app (C#) and https://wp8nativeaccess.codeplex.com/ for the native access wrappers (C++). You may have to fix up the reference paths to get the C# component to see the C++ component correctly. The code is reasonably well documented, but let me know if you have any questions. Permission to re-use the code or components is granted under the MS-PL (Microsoft Permissive License) as posted on Codeplex.
Go forth and find cool stuff!
Version history (see the git commit logs for more detail:
07 July 2013 - 0.2.0: Initial release, FS only, 920 downloads (source: 652 downloads)
14 July 2013 - 0.3.2: initial registry, HTTP server and web app encapsulation, source on Codeplex, 225 downloads
0.3.3: bugfixes, 454 downloads
0.4.2: basic registry values display, 86 downloads
0.4.3: bugfixes, 326 downloads
0.4.6: multistring registry values, bugfixes, updated libraries, first AllCapabilities version (950 downloads), 453 downloads
25 Oct 2013 - 0.4.8: binary and long registry values, formatting and bugfixes, 451 downloads AllCaps, 201 normal
22 Dec 2013 - 0.4.9: all registry value types, better threading, proper resume, remembers port, 97 downloads AllCaps, 53 normal
24 Dec 2013 - 0.5.0: background operation using Location APIs. Downloads: 1011 AllCaps, 963 Normal
20 Jul 2014 - 0.5.1: More capabilities, better navigation. Downloads: 358 AllCaps, 352 normal
07 Aug 2014 - 0.5.3: .REG export, better traversal, bugfixes. Downloads as of 0.5.5 release: 260 AllCaps, 164 normal
10 Oct 2014 - 0.5.5: Bugfixes and back-end work. Downloads as of 0.6.0 release: 140 AllCaps, 113 normal
25 Oct 2014 - 0.5.6: Bugfixes and UI tweaks. Downloads as of 0.6.0 release: 1720 AllCaps, 1334 normal
12 Oct 2015 - 0.6.0: Binary requests, file uploads, bugfixes.
XDA:DevDB Information
WebServer Native Access, Tool/Utility for the Windows Phone 8 General
Contributors
GoodDayToDie
Source Code: https://wp8webserver.codeplex.com/
Version Information
Status: Alpha
Created 2014-10-17
Last Updated 2015-10-12
I'm going to use this space to mention something that's pretty cool:
J. Arturo of http://www.komodosoft.net is using a modified version of the HTTP server that powers this app in the ShareFolder app (http://www.windowsphone.com/s?appid=e2b9c82e-eaa1-4a3b-9d4a-8a2933a8bdb4) to support opening video files directly from Windows network shares! This was done to work around a limitation of the WP8 media control: it can only source from an isolated storage file or a HTTP URL. By running a server in the background and streaming the video file through it, and pointing the video player control at the localhost URL, it becomes possible to play the file on the phone without first copying it to the app's isolated storage. A very cool way to solve the problem! Also, reviewing the changes that were made to the network code of the server pointed me toward those threading fixes I made that have hopefully much improved version 0.4.9.
Please note that the updated version of ShareFolder with this feature may not yet be available, although it should be soon. It is a commercial (paid) app, but the author sought and received permission to use my code (although the license does not require such permission be received).
What exactly is the problem with sockets? I am battling myself with sockets atm too, maybe we can share knowledge?
Strictly speaking, the problem was with the phone's limited subset of the Sockets API forcing me to access it through functions I wouldn't normally use (asynchronous everything, SocketAsyncEventArgs and lambdas and AutoResetEvents and so on everywhere...) but I've got a pretty good handle on it now, at least for the System.Net.Sockets.Socket and its friends. The new .NET 4.x ones (using the async keyword and all) are in a different namespace; I didn't mess with them. They are more abstracted from the Bekeley sockets interface that I'm used to from C, but they are also (supposedly) more user-friendly, especially if you don't feel like writing all your own thread management code (and in fairness, I should re-write the webserver's threading to use threadpools; they're better for this type of work).
If you want to ask questions about the topic, I suggest starting a new thread (possibly in the Q&A subforum, although it's also dev related...) and I'll answer if I can.
GoodDayToDie, just an idea: how about sharing your source code via CodePlex or GitHub?
Oh man, this is pretty nice! GoodDayToDie does it again!
So far, I can read \Windows, the current install folder which you access just by typing "." with no quotes and the current application folder by typing ".." I can access the .dlls, .winmd and AppManifest.xml from the current install, but from everywhere else, it goes boom. This is a great step towards something awesome though!
EDIT:
I was wrong. For some reason, when you click on a folder it's trying to "download" it, rather than chdir. I can get pretty far into the Windows directory.
THAT's what you meant by "Click on a file (note: there's no current way to tell the difference between files and folders) to download it.
You might see an error code (error 5 is "ACCESS_DENIED", you'll see it a lot). Or you might see a status 500 message because of an exception in the server. It's getting a lot more resilient but there are surely still some bugs. ".
If you see a folder, just type the full path to it instead of clicking on it and you will be able to read the contents.
ANOTHER EDIT:
I just found a file inside of the \Windows\System32 directory named [guid].devicemetadata-ms (It's easier to just search for "devicemetadata-ms"). It's a cab file with some metadata about WP8 with a sign.cat and packagesign.cat file in the archive. I don't know what these files could potentially be useful for.
New version in a day or two (busy tonight). Features I plan to implement (not necessarily in the next version or at any particular time):
File upload (IsoStore and, of all crazy things, install directory are writable. I think I'll put a flag on each FS page that says whether the current dir is writable...).
File deletion (where possible, of course).
File and Directory distinction in the listing (clicking a dir should open it, not error out).
Filesystem index page with links to folders that can be accessed successfully (since the root isn't readable).
Some more file info (size, probably attributes, possibly permissions).
Possibly an option to preview a file (as plain text) without downloading it.
Some kind of background mode (the server uses minimal resources when not actively servicing a request, so I'll see if I can get it to work in the background, perhaps by abusing the music transfer agent...)
Some kind of offline mode (at least basic file browsing within the app, as an alternative to using the web interface, though I might just make a second app for that).
Source code changes: separate the server code from the webapp / phone app code (move it into its own project).
Source code changes: move to a hosted version control service, probably CodePlex (good suggestion sensboston).
Maybe add an icon and such...
Any other suggestions?
I also want to try experimenting with various non-standard capabilities and see if I can get access to more of the system . I've already added the ability to access removable storage, but I've also found a bunch of really weird and frequently undocumented capabilities in the OS's policy configuration files, and I need to look into those... The interesting (and possibly the uninteresting) ones are probably blocked for unsigned sideloaded apps, but it's worth checking on anyhow.
Yeah sorry, I should have been more explicit about clicking on dirs. not working in 0.2.0. Also, it's "unofficial" but if you check the URL bar you'll see a URL parameter called something like "pattern" (by default, it's *) and if you change that, you can filter the results. For example, "foo*.exe" (note: no quotes!) will search for EXE files whose names start with "foo". Among other uses, this makes it a lot faster to load large dirs like System32. This will be added to the UI at some point. Also note that URL decoding is applied correctly to querystring parameters (Probably already noticed with the path sometimes written using %5C for \) so you can add special characters that way if needed, though currently any of them but \ will probably just cause an exception.
...
Actually, does this filesystem support Alternate Data Streams? If so, you should be able to download them by appending a : and the ADS name to the filename in the download URL...
OK, so that was a new version in five days. Sorry, stuff takes time.
The source code is now on Codeplex. The native access portion is at https://wp8nativeaccess.codeplex.com/, and the web server portion is at https://wp8webserver.codeplex.com/. Both are licensed MS-PL and use Git for version control. The full XAP is also available for download from the Webserver project on Codeplex.
GoodDayToDie said:
OK, so that was a new version in five days. Sorry, stuff takes time.
The source code is now on Codeplex. The native access portion is at https://wp8nativeaccess.codeplex.com/, and the web server portion is at https://wp8webserver.codeplex.com/. Both are licensed MS-PL and use Git for version control. The full XAP is also available for download from the Webserver project on Codeplex.
Click to expand...
Click to collapse
You are a god. I'll be sure to post my findings .
Hmm. When I first load up WebServer File Access then access from my laptop, I get the main page then the program crashes on my phone. It seems to hold a lock on to the socket as i can no longer access port 9999 from any other device when re-opening the app. I can access it again when I reboot, but the same thing happens.
EDIT: I think it may be due to the WiFi at work... it's junky. I'll try again when I get home. I was just able to browse some directories.
Wow, that's completely unexpected... I can beef up the error chacking and handling around the listener port though. That part of the code is really straightforward, so I actually haven't hardened it very much. I can also put in a Finally block to close the socket and/or mark the socket as re-usable so that other apps (or the same one again) can listen on it in the future.
I also plan to add support for setting your own port, but that doesn't solve the underlying problem. I'll put in more error reporting as well, to enable better debugging. Thanks for the report! Always good to have users report problems so I know where to prioritize fixes.
GoodDayToDie said:
Wow, that's completely unexpected... I can beef up the error chacking and handling around the listener port though. That part of the code is really straightforward, so I actually haven't hardened it very much. I can also put in a Finally block to close the socket and/or mark the socket as re-usable so that other apps (or the same one again) can listen on it in the future.
I also plan to add support for setting your own port, but that doesn't solve the underlying problem. I'll put in more error reporting as well, to enable better debugging. Thanks for the report! Always good to have users report problems so I know where to prioritize fixes.
Click to expand...
Click to collapse
I tried the app at home and it DOES crash on the first hit of the home page, but I'm able to open it up again and it works fine.
The new version 0.3.3 should be more rebust; try it and let me know if you still have issues. If you do, let me know what the exception message is (and any other info you can provide) and I'll try to track it down.
Downloading really big files should also work now. The app will read and push files in smaller chunks (the code to do this existed in the NativeAccess library before, but wasn't used).
a simple SDK?
Dear Sir
Will it be possible for you to make some sort of SDK from this so other developers can integrate this into their apps and enable browsing isolatedstorage?
Sorry if it is a stupid question.
Bruce_X_Lee said:
Dear Sir
Will it be possible for you to make some sort of SDK from this so other developers can integrate this into their apps and enable browsing isolatedstorage?
Sorry if it is a stupid question.
Click to expand...
Click to collapse
With the restrictions in permissions, this app only allows browsing of the app's isolatedstorage locally. You are able to use the IsolatedStorage API within your app to browse files and directories already.
snickler said:
With the restrictions in permissions, this app only allows browsing of the app's isolatedstorage locally. You are able to use the IsolatedStorage API within your app to browse files and directories already.
Click to expand...
Click to collapse
That's right. What I want is to allow the end user to be able to browse the isolatedstorage. Imagine I have a video download app, I want the user to be able to transfer those downloaded videos from the app's isolated storage to, say, a PC.
One can do this by integrating the webserver code into the said app.
Bruce_X_Lee said:
That's right. What I want is to allow the end user to be able to browse the isolatedstorage. Imagine I have a video download app, I want the user to be able to transfer those downloaded videos from the app's isolated storage to, say, a PC.
One can do this by integrating the webserver code into the said app.
Click to expand...
Click to collapse
Ahh I see what you mean now. That sounds like a pretty nice idea. I think more research needs to be done to see whether it would even be allowed in the marketplace.
The webserver portion is stand-alone (builds to its own .NET DLL with no dependencies on the other parts) and has a pretty clean interface. You'd need to implement the web application portion of it yourself - the thing that generates the response pages for a given request - but the HttpResponse class in the server does a lot of the work of that for you; you basically just specify the content you want to send (as a String or byte array) and it sends it.
I have a Shield TV arriving later today. On my previous Android box I setup a VPN with a kill switch via AFWall+. This involved the installation of the OpenVPN connect application with my VPN provider details, then I used AFWall+ to prevent any data that didn't originate via a VPN connection, effectively making this a kill switch. This involved allowing both the OpenVPN application and Android's VPN api full access. It's the latter I'm enquiring about, does this api exist on the Shield TV? I've heard that native VPN isn't possible as the normal VPN settings aren't present. But does that include the vpn service api itself?
No one?
Beefheart said:
I have a Shield TV arriving later today. On my previous Android box I setup a VPN with a kill switch via AFWall+. This involved the installation of the OpenVPN connect application with my VPN provider details, then I used AFWall+ to prevent any data that didn't originate via a VPN connection, effectively making this a kill switch. This involved allowing both the OpenVPN application and Android's VPN api full access. It's the latter I'm enquiring about, does this api exist on the Shield TV? I've heard that native VPN isn't possible as the normal VPN settings aren't present. But does that include the vpn service api itself?
Click to expand...
Click to collapse
i have the exact same setup on all my devices, including shield tv, although ive only had to allow the openvpn app, wifi/data/vpn access for things to work, ive never had to allow androids vpn ........is their a specific reason you grant android vpn access?does it not work otherwise?
I use the other openvpn app, by the way
I originally set it up on the tutorial in the link below, which mentions that the VPN Networking service needs to have full access. Is that service present on the Shield?
https://www.privateinternetaccess.c...otection-on-android-with-afwall-requires-root
Beefheart said:
I originally set it up on the tutorial in the link below, which mentions that the VPN Networking service needs to have full access. Is that service present on the Shield?
https://www.privateinternetaccess.c...otection-on-android-with-afwall-requires-root
Click to expand...
Click to collapse
I just checked for you, and yes, its there, mind you, im using zulu's full rom, not sure about stock rom but as with all my devices, i havent needed to allow this for vpn to work.
Unless theres a specific reason to do so, try without on your current devices, i suspect, vpn networking may only apply if you use androids inbuilt vpn found in settings
Edit
By the way, i dont know how far you wanna take it, but afwall has tasker plugin support, which i use to apply an afwall profile, i named "secure", that denies everything when screen turns off......aswell as other things in the same vain
Edit
I do it a little differently then what youre link suggests, i only allow the bare minimum of apps, those that i actually need internet for.......if an app has internet capability, but i have no need for that side of it, its denied, i dont whitelist ALL apps for vpn as your link suggests
I also suspect that guide was written for privateinternets method of using vpn on android, so maybe vpn networking applies if using private internet, but as for my openvpn app, its not needed.......neither is "GPS"
Cheers. Everything set up and working perfectly in stock, no DNS leaks. A combination of AFWall+, VPN and Xprivacy has the device locked down pretty well.
And what a device, the speed is in another league compared to other similar boxes and worth the extra money. I'm glad I returned my newly purchased Minix Neo U1, this thing is so much faster and not as restricted as I was lead to believe. With a bit of work the Shield TV, even on stock, can do as much as any other Android based TV box, even one based on vanilla.
Beefheart said:
Cheers. Everything set up and working perfectly in stock, no DNS leaks. A combination of AFWall+, VPN and Xprivacy has the device locked down pretty well.
And what a device, the speed is in another league compared to other similar boxes and worth the extra money. I'm glad I returned my newly purchased Minix Neo U1, this thing is so much faster and not as restricted as I was lead to believe. With a bit of work the Shield TV, even on stock, can do as much as any other Android based TV box, even one based on vanilla.
Click to expand...
Click to collapse
Yep, ive said it before and ill say it again, the shields an impressive piece of kit for sure
Xprivacy.........snap
We seem to have a very similar setup........believe me, if you wanna take it further at some point in the future.......tasker.........although, fair warning, theres a learning curve
Just some of the more basic things i automate with tasker with plugins like afwalls
When screen goes off, tasker......
Turns off wifi/3g
Turns of bluetooth
Afwall secure profile
Greenify all preselected apps
turn off "unknown sources" for extra measure, as tasker turns this off after it detects an apk install anyway
Turn of "debugging", incase i turn it on one day out of need and forget to turn off
Media volume set to 4 edit:this ones a bit out of place
Aplly afwall profiles depending on what app you happen to be using
Many possibilities with tasker, VERY usefull for many things
Non security related....kinda......... could potentially be used for such if modified
I have a small bluetooth media remote which has the numbers 1 to ten, with tasker and xposed additions module, i fooled around with it, pressing 1 connects the shields bluetooth to the bedroom speakers, long pressing 1 connects to the living room speakers..........i can imagine my self doing some neat stuff with these combination of apps and future accesories
Also, i use it to turn the shields light led to dim to let me now at a glance if the shields on or asleep, without having to change the channel
food for thought for those with similar setups
Edit
By the way, you mention dns leak, i assume you used a test site to check for the leak, any chance of a link? Incase its something very new
This ones the one i use,
https://ipleak.net/
Detects webrtc leaks on the specific browser you happen to be using at the time
Edit
For those interested
More on webrtc here
https://www.privateinternetaccess.c...ome-and-mozilla-firefox-while-using-private-i
If you use firefox or chrome, you can disable manually following this guide
https://www.purevpn.com/blog/disable-webrtc-in-chrome-and-firefox-to-protect-anonymity/
I think there are addons aswell
Edit
"and not as restricted as I was lead to believe"
Yep, i had the same thoughts, just my own assumption really, that android tv was completely different, internally, to "standard" android , pleasantly surprised, no incompatibilities so far............................good to know that stock is like that too :good:
Cheers, I'll read into all that.
One issue I'm finding at the moment is that, on a reboot, AFWall+ doesn't apply as default on the Shield and has to be done manually. This doesn't happen on my Note 3 running Lollipop. I'm sure there is a simple explanation, I'll look into it a bit more.
That website is the one I user to check leaks but there are numerous others too.
Beefheart said:
Cheers, I'll read into all that.
One issue I'm finding at the moment is that, on a reboot, AFWall+ doesn't apply as default on the Shield and has to be done manually. This doesn't happen on my Note 3 running Lollipop. I'm sure there is a simple explanation, I'll look into it a bit more.
That website is the one I user to check leaks but there are numerous others too.
Click to expand...
Click to collapse
Im not sure i understand fully, afwall is not enabled? Or, afwall IS enabled, but your prefered profile is not "applied"?
On full android at least, afwall is enabled upon reboot i havent had any issues in that regard, (saw your other post) i dont need init.d script (usefull to have though, if/when possible)
Have you tried reverting all afwalls settings to default, to rule out that likely suspect
Another likely suspect, xprivacy, but that depends if you restrict everything like i do, including system apps , if so, have you checked xprivacies usage data for afwall and global apps?
Another suspect, could be stock firmware, but i have my doubts about that one
Assuming im understanding the issue correctly
Edit
I dont have "fix startup data leak" checked(as we dont have init.d), nor ipv6 support checked as your link described
I am asking myself - specifically for the G5 Plus, but probably in a more general sense - where the huge advantages and disadvantages of rooting are, considering that the G5 plus comes with a relativly clean Android 7.XXX and a not an old overloaded android version, which didn't use to have many of the capabilities that Android 7 offers. I know that my questions might particularily overlap with questions in other topics, but for sure not every question, especially specific G5 Plus questions.
Overall I am interested in the topics security and product-experience, if you want to call it like that. I ask myself: Is root still worth losing warranty or is it not? Keywords or keyquestions that cross my mind are:
OTA updates: I guess those won't be possible anymore?
Encryption: Will it still work and increase security if the phone is lost?
Backup functionality, especially in combination with cloud services: Is there something like -backup my whole phone down to the very core on some google server (best proteced with a password and some AES256 encryption)- so that I can restore it some day in an easy manner? How would you backup your phone and settings, etc. with and without root?
Safety: What could happen if I lose my (bootloader unlocked and) rooted phone: Will someone be able to read my passwords (e.g. google...) and other sensitive information directly from the phone, even if it was locked, in the moment I lost it? What is the worst thing that could happen?
Root Functionality: How does the root access / superuser specificly work, e.g. if I'd accidentally install an app or similar, which might contain a virus: Is an app like this instantly capable of messing my whole system or will I be able to manually confirm specific security related changes, especially system changes, that an app might try to do? With other words: Does root mean that the system will be wasted by even the tiniest mistake or is there some security buffer?
Unlock Bootloader only: Is it an option (or make any sense to you) to just unlock the bootloader and install a the G5 Plus TWRP recovery without rooting the phone and does this give any advantages or is this just a totally nonsensical option, which is maybe not even possible? If I got it right, rooting does not necessarily need to reset the phone in any way, while unlocking the bootloader enforces to do a reset, right? In this context I was also asking myself if unlocking the bootloader (now that I don't have wasted precious time on customizing my phone, yet) right now is a useful option (without any disadvantage besides losing the warranty) and if I ever experience the necessity to root, I will only need like 2 commands and it is done - without having to reset my phone again?
Root Must Have: Is there any specific functionality or reason - you would say - one should definitly root the phone for, as it is a must have functionality, which would be locked without root?: I only have virtual examples, e.g. if Nougat would prevent me from changing the volume to a level higher than 50 % and the absolute exclusive possibility to change this was to get root access. Another example , although really not that critical one, could be: I noticed that I am only allowed to install 5 different finger prints... root could give me the possibility to install infinite finger prints?
Feature Loss: Does one lose some other neat features or functionality that is usually provided by Google or Motorola if the phone is not rooted but not possible anymore if it is rooted?
Third Party Trust: How can you people trust the TWRP Backup or custom ROMs? Don't you fear that there might be a virus or trojan horse within?
Best regards and thanks in advance for your patience with a newbie
No response?
172 view, no answers :-/. Guys tell me: Is it due to the length of the text? Is it something else? I could split it up in several questions, but I though that this would be unwanted.
And I will be thankful for every help on either of the bold buzzwords, it is not like you need to comment on everything
Must have for me: correct timestamps when moving or copying files using TC. Only possible with root.
Unlock only: yes makes sense. Unlock is the part where you lose all data, and then you can use fastboot boot to make backup. Rooting itself should not lose any data, so it is advantageous to unlock early. Root has time.
Lost functionality: on most devices using Magisk 12 you can pass SafetyNet, which means you can use Android pay, play Pokemon go etc, but the apps trying to detect root/unlocked devices get changed and may not work anymore at some time. Probably you will have lost this possibility when starting with unlocked bootloader and need to install Magisk to get green SafetyNet. Magisk hides the unlocked bootloader.
OTA: do a backup of boot partition before rooting, do no modifications on other partitions than data, cache and boot and you should be fine restoring boot partition to do OTA. It's easy to overlook some app using root to write system, logo, recovery, something, but backup should help. Or install complete firmware, then OTA is possible again.
Note: I do not have the device, just saw the questions which have the same answers for all current Motorola Android devices - you may search in general forums or forums for similar devices for answers
OTA updates: if you are rooted you have tempered with the system partition and therefore ota are not easily installed
Encryption:it is possible to wipe the phone and use if you are unlocked
Backup functionality Google already does backup some settings natively. you can still do an adb backup even without root
Safety: if they are techies they know how to access files via twrp etc. but worst thing is they just wipe it and use the phone
Root Functionality: root gives some apps access to the system partition which is not possible normaly. if you installe some dubious app which wants access to root to mess with your system you are lost.
Unlock Bootloader only: you need to unlock the phone to root it. by unlocking your phone is wiped clean. than you can root it. the advantage of installing twrp are the "easy backups" and installing custom roms or even root. there are no real advantages or disadvantages anymore. earlier you had to unlock/root/install custom rom to have some extra functionalities but android did mature and has most functions built in
Root Must Have: there may be some system limits which you can bypass with root like headphne volume limit, reading wifi passwords or/and having systemwide adblock. I personally do not see a benefit anymore. I used to root for having system-wide adblock but I can achieve it with rootless apps like adguard.
Feature Loss: you will lose android pay. you can not use some apps like mario run or pokemon go. you will lose OTA feature.
Third Party Trust: actually I dont know. with the custom rom base growing I only trust official lineageOS as it is review by many people before building. therefore the chance is reduced to have some spyware feature in it
I too would like to know, has the source code to ANY custom ROMs been reviewed by third party to verify no malicious code?
Although I worry that some ROMs could violate my data privacy, root is something that I simply cannot willingly go without - if I don't have root access, it's simply not *MY* phone, it's a phone that is configured to someone else's [some company's] desires and priorities.
I'm disappointed that the built in tethering does an "entitlement" check - AFAIK it's actually illegal (or, at least against contracts the companies signed with the FCC) for the cell phone provider to attempt to control what a user does with their allotted amount of cell data. Yes, the cell provider company can decide how MUCH data you are allowed based on what plan you pay for, but they are not supposed to restrict HOW you use YOUR data. Therefore, I demand unrestricted "tethering" from any smart phone that I use.
There are other apps I like to use that require root access: Root file explorers, Titanium Backup, Smarter WiFi Manager, Greenify/Servicely etc., but most of all, I CANNOT STAND the intrusive obnoxious awful ads which seem to be prevalent these days! A good ad blocker is an absolute must! The blame rests squarely on the shoulders of the websites which allow such awful advertisements such as "pop behind" windows and particularly, ads which cause the web page scroll to constantly keep jumping away from what you are trying to read making the site basically unusable. There is also lately a prevalence of "click bait" ads/links which brings you to malicious/obnoxious websites which popup dialogs trying to stop you from closing the web page or navigate away - they put up big flashing red letters and say things like "We have detected a virus on your computer do not close this window or your passwords will be stolen and your data lost" and when you try to close the page it keeps popping up a dialog making it difficult. Sorry, but, such ads simply can't be tolerated - even this [xda] website sometimes has unpleasant ads, or at least there were times when I really regretted turning off my ad blocker when visiting this site in the past, that is for sure!
I usually use a "custom ROM", I miss exposed very much, but, I suspect there are too many malwares in the xposed repository these days? (I'm not sure of this, just suspicious).
I like to be able to change the color of my status bar clock to green and position it in the center as that is easier for me to use (see it quickly when I want). However, the standard launcher is far too limited in how customizable it is, so I use a combination of Nova Prime (requires root for some features) and Chronos Weather/Clock/Calendar widget which puts a larger clock right in the upper middle of my desktop so I turn off the status bar clock (Nova Prime feature, one that requires root).
Oh, and I like to use a custom "System Font", I'm not sure if we can do that without root? It really makes the phone feel like MY phone and look (and operate) how I want it to.
critofur said:
[...]
I'm disappointed that the built in tethering does an "entitlement" check - AFAIK it's actually illegal (or, at least against contracts the companies signed with the FCC) for the cell phone provider to attempt to control what a user does with their allotted amount of cell data. Yes, the cell provider company can decide how MUCH data you are allowed based on what plan you pay for, but they are not supposed to restrict HOW you use YOUR data. Therefore, I demand unrestricted "tethering" from any smart phone that I use.
There are other apps I like to use that require root access: Root file explorers, Titanium Backup, Smarter WiFi Manager, Greenify/Servicely etc., but most of all, I CANNOT STAND the intrusive obnoxious awful ads which seem to be prevalent these days! A good ad blocker is an absolute must! [...]
[...]
Click to expand...
Click to collapse
Could you explain the entitlement check a little further? Does it mean that with the current Android version and an unrooted/locked G5 plus it is impossible to use the Smartphone Mobile data connection, e.g. on a notebook via wifi tethering? This would be a real argument to root.
Did you try adguard, as ckret suggested? Is there a huge difference between an adblocker with root or an adblocker like adguard without root on the phone? I basically assume that with nougat it is possible to grant apps access to almost anything (except for root) - including to block features other apps use, e.g. advertisements. But I am actually not sure.
Maybe ckret knows more on this aspect, as he seems to know both adblock concepts - the rooted and the unrooted one with adguard?
Comparing DNS66 (local DNS server without root) with adaway (root):
+ You can select blocking per app with DNS66, adaway modifies hosts file which always is valid for all apps and system services
- You can not use another VPN while DNS66 is active
- You need to disable VPN under Nougat while using Download Manager (bug in Nougat, for all VPN services)
Personally I have root, but use DNS66. I don't need adblock when connecting to my computer at home (that's when I need to use another VPN) and am using Marshmallow ATM, but probably would continue using DNS66 when on Nougat. For PlayStore there is a workaround implemented, and if some download fails I'd know I need to disable VPN.
This is why I only said Total Commander copying timestamp is my only real killer app (besides Titanium Backup) which makes me need root. Android O is supposed to change the behavior implementing SDCardFS which shall allow setting timestamp without root.
sky-head said:
Could you explain the entitlement check a little further? Does it mean that with the current Android version and an unrooted/locked G5 plus it is impossible to use the Smartphone Mobile data connection, e.g. on a notebook via wifi tethering? This would be a real argument to root.
Did you try adguard, as ckret suggested? Is there a huge difference between an adblocker with root or an adblocker like adguard without root on the phone? I basically assume that with nougat it is possible to grant apps access to almost anything (except for root) - including to block features other apps use, e.g. advertisements. But I am actually not sure.
Maybe ckret knows more on this aspect, as he seems to know both adblock concepts - the rooted and the unrooted one with adguard?
Click to expand...
Click to collapse
adaway:
adaway replaces the hosts file in your system with a custom hosts file which redirects some requests to 127.0.0.1 which results in ads not being shown
since it is deeplevel change of the hosts file the app requires root to change the file
pro:
* ads are blocked when resources are requested
* it is system-wide and everything is checked on demand
con:
* system slows down with big hosts file as every request must be checked everytime a site/app is opened
* if a wrong request is blocked your app/site might not show/work at all since it is a system-wide check
adguard:
this app has two different ways of blocking ads
vpn: a local vpn server is created on the system and all requests are rerouted through it. works the same way as adaway but without a root access.
pro:
* rootless method
* you can create a bypass for different sites/apps
con:
* you can not use a 2nd vpn connection while the app is active
* it may use a bit more battery as it creates a server but this should be negligible
proxy: this is nearly the same as vpn just you should be able to use a vpn connection
so big pro and con for me is that i do not have to reroute all apps through the adblock check
important apps (banking e.g.) are free to use the connection without being rerouted.
I know it might seem like a stupid question, but how often (and for which reason) do you use/need a(nother) VPN connection?
Does this also mean things like tethering or a WLAN access like eduroam - or is this something different?
I am actually not sure if I ever needed VPN on my smartphone
sky-head said:
I know it might seem like a stupid question, but how often (and for which reason) do you use/need a(nother) VPN connection?
Does this also mean things like tethering or a WLAN access like eduroam - or is this something different?
I am actually not sure if I ever needed VPN on my smartphone
Click to expand...
Click to collapse
you need a vpn connection if you want to access the intranet without being physically there
e.g. intranet of a company to access emails or if you are a student and got some special tool/e.g. which can only be accessed through the university connection
most times you will only use vpn on a notebook or pc but I hardly doubt most people will use it on their phones
ckret said:
you need a vpn connection if you want to access the intranet without being physically there
e.g. intranet of a company to access emails or if you are a student and got some special tool/e.g. which can only be accessed through the university connection
most times you will only use vpn on a notebook or pc but I hardly doubt most people will use it on their phones
Click to expand...
Click to collapse
... exactly what I was thinking about it. I've never been needing a VPN on my phone. On the notebook I need it on a regular basis, thats true.
I should have been asking "I know it might seem like a stupid question, but how often (and for which reason) do you use/need a(nother) VPN connection on your smartphone?", to state my question more precisely.
Using AVM Fritzbox as router makes it possible to use the standard phone via SIP. This only does work when you're in your intranet, directly or via VPN. Also I need to access my documents on my computer, my media library at home, to configure the router and more and therefore I use VPN on a regular basis. Yes, I do these things using the smartphone. But when using VPN, I do not need adblock.
I'm reading packets in native code from the TUN interface created with the VpnService API. I would like to correlate packets to installed applications, i.e., to know which application sent a certain packet, without root access. With root access it would be a simple case of either using netstat/lsof or going through some of the /proc files. However, I couldn't find a way to get a list of connection <--> PID (or UID) mappings, neither in the native Linux context, nor within the higher level Android APIs.
I don't mind a more convoluted solution that needs work and is somewhat hackish, as long as it works without root access.
Looking forward to even ideas and starting points that I can further explore myself, if you don't have a full solution. And if you know 100% this is not possible, no matter the workarounds that I may try, let me know.
To answer my own question, in case someone finds this post with a search engine, yes it is possible, at least up to Android P, by reading the /proc/net/tcp, /proc/net/tcp6, /proc/net/udp, and /proc/net/udp6 files. However, in Android P the ability to read files under /proc/net is starting to be restricted (see this thread) and is going away in Android Q.