[RELEASE] beta.exe.1.1.1 | Automatic Captions and Censors

[theredreaper]

bonsec if you would PM me at your convenience, there is a major security issue with your application that really needs to be addressed. I took the liberty of running your pyinstaller package through a decompiler and poking around a little bit under the hood, i hope you don't mind. I just didn't want to raise this kind of an issue without being able to explain where it comes from and more importantly, how to fix it. I also think there are some easily attainable performance gains you can implement without much trouble, along with implementing a couple of small changes that will address some feature requests I would make, and I'm sure others will be interested in. If you're not keen on the unsolicited consult, I understand, but we DO need to talk about the security problem, because it's a concerning one.

[theredreaper]

I have yet to hear back from the OP on this, so I guess it's time to go public with the vulnerability it exposes. I did a bit of testing to verify that is indeed as bad as I thought. I wanted to give him a chance to fix it before revealing the flaw, but it's serious enough that I cannot, in good conscience allow any more time to pass that people might be using this software.

The major issue is with the web server he's using. You may have noticed the console window that opens up in the background with the warning message about not using this web server in a production environment. The reasoning is because of the way it opens ports to allow for TCP communication. Most consumer grade routers these days have a technology called UPnP that is generally enabled by default. UPnP offers a lot of user convenience, because it can automatically open ports that are necessary for certain programs to communicate on, saving the user from having to manually configure port forwarding settings to allow their apps to communicate outside the network. However, this web server script that he's using is an extremely rudimentary one that is built into one of python's libraries, intended only for quick testing and demonstration purposes, and it has no kind of security to discriminate about who can communicate on these ports. So in many cases, combined with UPnP running on a router, it's going to expose an unencrypted TCP port to the Internet. Worse yet, many consumer grade routers don't even implement UPnP properly, or have bugs in the implementation. I have seen everything from exposing ALL ports on a host that requests any ports, and stupid features like "service matching" that will redirect ANY traffic with a given protocol to any port equipped to answer it, and many other issues. Any given router is probably being port scanned by as many as a dozen different automated programs at any given time these days. These bots are simply iterating through an entire block of IPs at a time, looking for open ports that expose services that are easily exploitable. It doesn't get any more exploitable than a wide open, unencrypted web server with no kind of password that doesn't even log its traffic. Now, if that was the only issue, this wouldn't be as concerning as it is, because for the most part, access to this web server running on a personal computer that isn't hosting any important data, while its an extremely trivial attack vector, generally doesn't provide access to anything that would be of concern. But this program exposes a native file explorer dialog window at certain points. In Windows, within the native file explorer window, you have all the capabilities of the explorer.exe process available through a right click context menu, which means anyone with access to this web server can now download, execute, copy, delete, etc any file that is available to the web server itself. Usually, a web server program would run as its own special class of user in Windows, but this is an extremely basic web server that simply runs as the user who launched it, which means at that point, a remote attacker has the same level of access to your computer as you do. They could even do something like uploading a shell of their own to install a more permanent "back door," or install pretty much any kind of malware they feel like. They could access files anywhere on the PC, and essentially operate it just as if they were sitting in front of it. And if you happen to be one of the many Windows users that has turned off User Account Control because it's annoying, this vulnerability will happily elevate the remote attacker's credentials to Administrator for the asking.

Now, I really like the idea that he has, but the implementation is slipshod, inefficient, and frankly irresponsible. I have the decompiled code and I was going to attempt to fix it, but honestly, it's so bad, I've decided to just start over. I already have quite a bit of the basic functionality banged in, and everything is user configurable from the captions to the amount of censoring or whether censoring is even applied at all. I have a few other plans to implement some features that I think people will find useful or enjoyable. I should have an alpha release candidate in a day or two. The codebase will be wholly original, so I will be claiming it as my own work, but I will credit OP for the concept.

[devotes.julchen]

Now, I really like the idea that he has, but the implementation is slipshod, inefficient, and frankly irresponsible. I have the decompiled code and I was going to attempt to fix it, but honestly, it's so bad, I've decided to just start over. I already have quite a bit of the basic functionality banged in, and everything is user configurable from the captions to the amount of censoring or whether censoring is even applied at all. I have a few other plans to implement some features that I think people will find useful or enjoyable. I should have an alpha release candidate in a day or two. The codebase will be wholly original, so I will be claiming it as my own work, but I will credit OP for the concept.

I'm also a big fan of this software idea and I'm glad you're embracing it. When using the current software from bonsec, I always had security concerns. And I am glad that you are now taking care of another implementation. If you are interested in offering the software in other languages, I would be happy to translate it into German.

[wqslave]

Now, I really like the idea that he has, but the implementation is slipshod, inefficient, and frankly irresponsible. I have the decompiled code and I was going to attempt to fix it, but honestly, it's so bad, I've decided to just start over. I already have quite a bit of the basic functionality banged in, and everything is user configurable from the captions to the amount of censoring or whether censoring is even applied at all. I have a few other plans to implement some features that I think people will find useful or enjoyable. I should have an alpha release candidate in a day or two. The codebase will be wholly original, so I will be claiming it as my own work, but I will credit OP for the concept.

Wow. Just chiming-in to say thank you, and that I'm excited to see what you release in the coming days.

[bonsec]

Thanks for the detailed comment!
I agree the webserver part isn't well implemented.
About the security concern, the port is only open to local network and not to the internet.
On the routers I've used, opening a port to the internet requires explicit configuration to do so.
How did you test being able to connect from outside the network?

Excited to try your release by the way! =)

[theredreaper]

Now, I really like the idea that he has, but the implementation is slipshod, inefficient, and frankly irresponsible. I have the decompiled code and I was going to attempt to fix it, but honestly, it's so bad, I've decided to just start over. I already have quite a bit of the basic functionality banged in, and everything is user configurable from the captions to the amount of censoring or whether censoring is even applied at all. I have a few other plans to implement some features that I think people will find useful or enjoyable. I should have an alpha release candidate in a day or two. The codebase will be wholly original, so I will be claiming it as my own work, but I will credit OP for the concept.

I'm also a big fan of this software idea and I'm glad you're embracing it. When using the current software from bonsec, I always had security concerns. And I am glad that you are now taking care of another implementation. If you are interested in offering the software in other languages, I would be happy to translate it into German.

My implementation will load captions from a user-provided text-file. I will probably distribute a "default" file along with it, but it's hard to imagine coming up with any collection of that sort of thing that works for everyone. When it's finished (I'm 75ish percent there by now), I will most likely just open source it and release it to the community, along with the source code, so you will be more than welcome to contribute translations, more captions, etc to the project. I don't intend to do much more after the initial release than whatever bug fixes might become necessary, as I have already been contacted by a couple of the other developers on this forum about getting involved with some of the bigger projects going on here, and am quite excited about that.

[theredreaper]

First, let me apologize for the tone of my last post about this, I knew the second I saw the console window open up with that familiar warning that it was potentially problematic, and once I confirmed the likelihood of real danger to an unsuspecting user, let's just say I got a little... excited. There's always a place in this scene for amateur developers, and as someone with many years of experience in both software development and infosec, I should be reaching to mentor rather than leaping to judge. So all that said, you have my apologies for previous criticism.

Now, I will respond to your queries with a cooler head.

About the security concern, the port is only open to local network and not to the internet.

Are you certain about this? It's not your code, after all. And I couldn't find anything in your program that looked like an attempt to ensure that this was the case.

On the routers I've used, opening a port to the internet requires explicit configuration to do so.
How did you test being able to connect from outside the network?

Certainly, as I commented, not all routers are created equal. Specifically, I am pointing to the real garbage grade "consumer" routers, stuff made by companies like Comtrend and usually integrated with something like a DSL or cable modem. If you're a "computer guy" of any measure, and I wager you are, then I likewise suspect that you aren't using such a router. However, we find UPnP implementations that are problematic across the spectrum. Partially it is manufacturers doing what they can get away with because the average consumer "just wants it to work" and doesn't know or care what UPnP is to start with. But they don't want to muck around with port forwards or NAT settings so Junior can play on Xbox Live. So the solution is UPnP, or in many cases... something that sort of resembles it. The thing about UPnP is, it's not a specific protocol per se, rather a kind of framework that establishes that devices should respond to certain requests in certain ways, and in the case of a router, answering UPnP requests generally means port forwarding. So most consumer grade routers will respond to any broadcast on UDP port 1900, identify themselves as a UPnP server and respond with a bit of XML containing their "menu" of services so to speak. From here, a device just needs to let the router know that it's expecting traffic on port xyz and protocol x, and the router handles the rest. The specifics of how all of these steps are handled, vary widely from firmware to firmware and from device to device. The whole point of UPnP is to be a zero-config system. Do you have to access your router to explicitly open ports for say, every game you play online, for cloud-sync operations, to get your Xbox or PlayStation to connect to its services, for Windows to get updates, etc? I'm guessing the answer is no. Thank UPnP and it's zero-config model. So while in an ideal world, it WOULD require explicit configuration every time we opened a port to the internet, end users in this ideal world would eat hardware manufacturers alive every time they had to touch the router config. To most people it's a black box full of magic smoke and they just want it to work. And the definition of "working" is pretty broad. They want anything they connect to that box to just do its "stuff" without giving them any hassles. So the farther removed from enterprise grade a particular piece of equipment is, the more likely it is to have a very permissive UPnP setup, and have it enabled by default. The worst offenders, in order to attempt to provide any functionality a user might want, without requiring any finicky configuration, do something called service matching. Simply put, the router sees a web server serving http traffic behind it, and it just assumes that server is expecting traffic from the web, and happily forwards anything coming in on port 80, OR anything carrying http headers OR anything directed at whatever port that web server happens to be configured to operated on right on through the firewall, because requiring the user to have knowledge is considered the bigger crime.

Anyhow, as far as my testing methods, pretty simple. I ran your script on a virtual machine on my home server, and passed it its own NIC, and let it take a DHCP IP address. I have a static IP due to the fact that I run a few different web servers and do some managed IT services work out of my house, so I already had that information. As you would imagine, with my normal router (A Fortigate F1000-D) running, there was no answer on port 2021, where your app runs its web server. But when I disconnected that router briefly and went ahead and ran my WAN connection into some random Netgear router I had laying around, I was able to access the VM from my laptop, which was tethered to my cellphone at the time, putting it outside of my LAN and ensuring that this request was being made via the internet. The Netgear unit was all too happy to pass my request on 2021 right along to the VM, and so was another consumer grade wireless router made by TrendNet, and 3 out of 4 Linksys routers. (The 4th, I discovered was flashed to DD-WRT).

As far as what to do if you want to fix this, while serving the app over HTTP does have its (limited) usefulness, I would question whether it's strictly necessary. If the reasoning for this was so you could design the UI with a flask template rather than doing a GUI, I would direct you to check out a library called PySimpleGUI. I know GUI programming can be daunting, but with this library, you can get everything you need up and going in less than 20 lines of code. If you're stuck on the idea of serving it over the network, study up a bit on sockets in Python, and implement your own server/client protocol. Or at the very least, put some kind of password controlled access in place AND remove the call to the native file browser. Instead, replace it with something you're in control of, and confine it to your application's working directory. I think the vast majority of end users are going to be satisfied with simply using the app on whatever device they install it on, and would be horrified to discover that there was even a remote possibility they could be exposing their porn collection, let alone their entire computer to incoming traffic from the Internet. Even if one had the utmost confidence that access was restricted to the LAN, are you familiar with how easily WPA2 WiFi keys can be cracked these days? All it takes is for the neighbor kid to be watching a couple YouTube videos about besside-ng and nmap, and something like this running on your LAN could be under discussion at your neighbor's breakfast table. Now, of course, we can do thing like MAC filtering or WPA3 keys to try and secure our home WiFi, but the question is, what percentage of the people who are likely to be interested in this software would be aware of the need for caution and/or have the skills to implement such measures? Let's be honest, when we're sitting down at our PC to fire up something like this, our minds are... elsewhere.

[bonsec]

Thanks for taking the time to explain in detail.
If I understand right, the problem is when:
* public ip accessible from internet (depends on ISP and service plan);
and
* router that will automatically "service match" if it detects a webserver.

I couldn't find much information about those types of routers so I'm unclear on how widespread this situation actually is. I update first post for now to add a warning and potential workaround about this security flaw for awareness.

As for why the app is a webserver, the reason is because I like to use my phone browser when I play with it.

[theredreaper]

No. Whether or not you have a static or "public" IP is irrelevant.

You have a WAN-facing IP no matter what. It may be a dynamic IP, but that doesn't matter for this particular scenario, because we are talking about a "right now" type of vulnerability. If your router has a traffic logging feature, check it out sometime, and just see how many incoming connection attempts you are getting per minute. Lots, right? Those are bots doing port scans on blocks of IPs, searching for open ports with easily exploitable services. The issue is that this type of web server is a big shiny BEACON to those bots saying, hey, come on in and install whatever you want! If any of that traffic happens to be an actual person who's actively scanning IPs, well then the situation goes from bad to worse.

As I more or less demonstrated, we are talking about the MAJORITY of consumer grade routers. 5 out of 6 that I just happened to have on hand all opened the port to the WAN side as soon as the web server came online. I mentioned service matching as one of the most egregious offenders but it's not the only thing that exposes this vulnerability. That is why the documentation for the http.server call and the console both give you a warning in BIG CAPITAL LETTERS, don't use this in production. Its intended to pop up a simple http server for extremely short term purposes, or for testing/demonstration purposes, not to be used in a program you're going to release.

[wqslave]

No. Whether or not you have a static or "public" IP is irrelevant.

You have a WAN-facing IP no matter what. It may be a dynamic IP, but that doesn't matter for this particular scenario, because we are talking about a "right now" type of vulnerability. If your router has a traffic logging feature, check it out sometime, and just see how many incoming connection attempts you are getting per minute. Lots, right? Those are bots doing port scans on blocks of IPs, searching for open ports with easily exploitable services. The issue is that this type of web server is a big shiny BEACON to those bots saying, hey, come on in and install whatever you want! If any of that traffic happens to be an actual person who's actively scanning IPs, well then the situation goes from bad to worse.

As I more or less demonstrated, we are talking about the MAJORITY of consumer grade routers. 5 out of 6 that I just happened to have on hand all opened the port to the WAN side as soon as the web server came online. I mentioned service matching as one of the most egregious offenders but it's not the only thing that exposes this vulnerability. That is why the documentation for the http.server call and the console both give you a warning in BIG CAPITAL LETTERS, don't use this in production. Its intended to pop up a simple http server for extremely short term purposes, or for testing/demonstration purposes, not to be used in a program you're going to release.

So I'm next to clueless about network-anything which is why I don't touch it when I code myself, but just for the sake of clarity, there's no real risk to a webserver like this if you are not actually connected to the internet, right? Like if it's just a router, not a modem, and you have no internet connection. Beyond like, someone literally outside your house connecting to your router somehow.

Is that (offline-only router) the environment that this sort of solution is intended to be used in?

I was under the impression that this sort of attack still requires ie browsing and having code forced into a popular standard port, like described here ( https://cathyjf.com/articles/local-serv ... ompromised) with the example of

Code: Select all

<img src="http://localhost:8080/vulnerable.php?p=arbitrary-code-here">

[theredreaper]

If you're not connected to the Internet, then you are correct in that the vulnerability is not really a problem, apart from the case you suggested where someone is literally hacked into your WiFi.

I was under the impression that this sort of attack still requires ie browsing and having code forced into a popular standard port, like described here ( https://cathyjf.com/articles/local-serv ... ompromised) with the example of

The reason this particular issue is so concerning is that it eliminates the need for any such phishing attack, and rather exposes a highly exploitable service to the internet where automated systems are constantly scanning for just such a thing to show up.

[wqslave]

If you're not connected to the Internet, then you are correct in that the vulnerability is not really a problem, apart from the case you suggested where someone is literally hacked into your WiFi.

I was under the impression that this sort of attack still requires ie browsing and having code forced into a popular standard port, like described here ( https://cathyjf.com/articles/local-serv ... ompromised) with the example of

The reason this particular issue is so concerning is that it eliminates the need for any such phishing attack, and rather exposes a highly exploitable service to the internet where automated systems are constantly scanning for just such a thing to show up.

Gotcha, thank you for the "explain like I'm 5" version haha, again anything related to the internet is so far beyond the scope of my programming experience or just computer science knowledge. That is scary and unfortunate /:

[devotes.julchen]

I get an error when I try to open the gallery

Code: Select all

ERROR:webapp:Exception on /gallery [GET]
Traceback (most recent call last):
  File "flask\app.py", line 2447, in wsgi_app
  File "flask\app.py", line 1952, in full_dispatch_request
  File "flask\app.py", line 1821, in handle_user_exception
  File "flask\_compat.py", line 39, in reraise
  File "flask\app.py", line 1950, in full_dispatch_request
  File "flask\app.py", line 1936, in dispatch_request
  File "webapp.py", line 117, in gallery_saved
KeyError: 'save_folder'

[Blackpanter]

Hi,

first its a great software.

is there any way to save a censored video.
If i Censor a video i can only save pictures of the video or can i only play the video ?