[GuideMe] Tease Downloader with EOS Support

[PlayfulGuy]

As noted in this thread, recent site maintenance seems to have broken the Downloader.

Patiently waiting and appreciative of your time to release a fix, PlayfulGuy.

I've been working on this with help from FrozenWolf and mostly thanks to him we have it figured out. I'm sick with a cold and not too energetic today, and as always these upgrades tend to snowball a bit, but a new version should be out by the end of the week, including new features and previously requested fixes, like that library issue.

Stay tuned!

[PlayfulGuy]

New release - version 0.97

I've uploaded a new release fixing download errors caused by recent Milovana updates, and most issues reported since the last update.

Big thanks to FrozenWolf for his help in getting the download parts working properly again!

The new version is unfortunately flagged as a virus/trojan by a few AV programs, but there's nothing I can do about that. See the results screenshot below, and the details here.

Some highlights

• Fixed many things that caused crashes
• The tease title, filename and media folder can now be edited in the GUI. The new values will be saved in the cached script file (if that option is on).
• Fixed issue with EOS teases not appearing in the GuideMe Library if no image exists in the root of the media folder
• In EOS tease "say" actions the text is now enclosed in double quotes so it's easier to edit text containing words that use single quotes like "I'm" and "you're"
• And you know what, I'll post a more extensive list in the next post.....

Or for more details see "Release Notes.txt" in the source distribution. (Next time I'll put those in the executable zip).

Known Issue:

• If the tease name contains an ampersand (like in "Tease & Denial") the downloader downloads and
  converts it successfully, but GuideMe does not properly load images because of the ampersand in
  the media directory name. To fix this change the name of the directory (like to "Tease and Denial") in the GUI before downloading.

As always the download links in the first post of this thread have been updated. Get your copy today!

PG

MiloDownloader_0.97 Virustotal Result.jpg (132.93 KiB) Viewed 7614 times

[PlayfulGuy]

Downloader version 0.97 Release Notes


Bug Fixes

• Fixed issue with downloader not working after Jan 2024 updates to Milovana.com
• Fixed issue with EOS teases not appearing in the GuideMe Library if no image exists in the root of the media folder. The downloader will now find an image and add it to the root of the media folder.
• Fixed bugs in the javascript to resolve issues in some converted EOS teases.
• Fixed a bug in batch downloading. If a batch download failed due to an error or was cancelled partway through then resumed later, the options to keep or overwrite existing XML files when restarting the batch did not always work as expected.

New/Changed Features

• The tease title, filename and media folder can now be edited in the GUI before downloading.
• As long as the option to cache local copies of scripts is on (see the Options tab) the downloader will now remember the folder you download a tease to, as well as the tease title, filename and media folder name used.
  If the download is interrupted and resumed later, or if you attempt to download the tease again, the download folder used previously will be restored and used again. If you have moved or renamed the tease after downloading you can update the download location, filename etc. before downloading again.
• The URL's used to retrieve pages from Milovana are now configurable strings. See the URLs tab.
  If these strings are changed in the future because of site changes at Milovana you can just update them on the URL's tab.
  In the event Milovana is down, it might also be possible to enter URL strings that will retrieve the tease information from the wayback machine, but I haven't tested that yet.
  Edit the URL's as needed, then click Save to save the new settings. If you don't click Save the URL's will be in effect only until you close the downloader. You can click the "Reset to defaults" button to restore the default strings.
• Batch Downloads: When downloading a batch of teases the download folder selected in the GUI at the time the batch download begins will be used for all teases in the batch. If a tease in the batch was previously downloaded to a different folder, it will be downloaded again to the batch folder.
  I am not 100% sure this is the best default. Feedback would be appreciated on that.

Changes to EOS Tease handling

• In say actions the text is now enclosed in double quotes so that it's easier to edit text containing words that use single quotes like "I'm" and "you're".
• Say actions now support text alignment options (left/right/center)
• Prompt and Option actions support an optional "dontShow" property. Normally the prompt response or button text is added to the dialog. This option suppresses that so the response does not appear in the dialog.
• In previous versions "say" text with mode "autoplay" that was longer than 50 words in length was changed to a mode of pause. In this version I left it as timed with a delay of 25 seconds.
• The downloader now always attempts to detect a BPM value for audio actions, and always adds the "bpm" property. This way if a tease is downloaded with one option, you can always do a global search & replace to replace the audio actions with metronome, or vice versa, and everything will still work. The default BPM used if none is detected can be set on the Options tab.
• Changed handling of a say with mode instant followed by a goto:
  In previous versions the commands were merged and the say was converted to mode "pause" and a warning was logged.
  In this version the commands are merged, the mode is changed to "timed" with a delay of 5 seconds, and an informational message is logged. This makes the behavior more like what was probably intended.

New options for EOS teases (on the EOS tab)

• Convert timed say followed by choice or option to mode choice.
  When selected this option eliminates the delay between the say action and the appearance of the button(s)
• Emulate GuideMe global buttons (default is on)
  When this option is selected GuideMe global buttons are emulated in javascript to work around some bugs in the current implementation of global buttons in GuideMe. When this option is selected the 'Simplify pages when possible' option is ignored and pages will not be simplified.
• Paused say includes "timed" properties
  When this option is selected a say action that pauses will also include the required properties for a "timed" say action so it can be easily converted by changing the "mode" from "pause" to "timed" and adjusting the delay.

New options on the XML tab

• Keep the start page first - always writes the start page as the first page of the guide (default is true)
• Sort pages by page name - sorts the pages of the tease by page name (default is true)
• Ignore case - ignore case of page names when sorting by name (default is true)

[pagengifflarkanel]

Thanks so much for all the time and effort you put into the downloader! I love the functionality of it and use it a lot to be able to play the teases offline. Very happy to have it working again.

I did have some small issues getting it working for me, not sure if it's just specific to me or perhaps someone else might run into the same issue.

When I first tried to run the program it would give me an error saying that the following file/directory was not found:

<snip>

The error message in full was:

Traceback (most recent call last):
File "MiloDownloader.pyw", line 512, in <module>
File "DownloadLib.py", line 439, in get_tease_script
File "DownloadLib.py", line 1209, in fetchPage
FileNotFoundError: [Errno 2] No such file or directory: '<snip>'

I did open the program from the <snip> drive but I dont have a <snip>' folder on there. I couldn't get it to open since it would give me the error and by clicking "ok" it would close down the program. So I just manually created the following folder on my <snip> drive and the program would then run.

<snip>

After opening the default download folder was the before mentioned <snip> so I changed that to the regular folder I normally use and tried to download a tease but it wouldn't work and it would give me the following error:

Traceback (most recent call last):
File "MiloDownloader.pyw", line 555, in <module>
File "DownloadLib.py", line 548, in get_tease_script
File "DownloadLib.py", line 676, in fix_Guideme_library_issue
KeyError: 'files'

I then tried it without changing the default download folder but it would still give me a error. I also tried it with changing or not changing the "Cache" folder on the Options tab but to no success.

I then decided to be a smart software tester and I opened the program again, changed the download default folder (and "Cache" folder) to my prefered folder and tried a different tease and that one worked. Tried a few more teases and they all worked.

Went back to the first tease I tried and that one still doesnt work so I'm guessing the error is specific to that tease (https://milovana.com/webteases/showtease.php?id=68913)

[PlayfulGuy]

Thanks so much for all the time and effort you put into the downloader! I love the functionality of it and use it a lot to be able to play the teases offline. Very happy to have it working again.

I did have some small issues getting it working for me, not sure if it's just specific to me or perhaps someone else might run into the same issue.

When I first tried to run the program it would give me an error saying that the following file/directory was not found:

<snip>

The error message in full was:

Traceback (most recent call last):
File "MiloDownloader.pyw", line 512, in <module>
File "DownloadLib.py", line 439, in get_tease_script
File "DownloadLib.py", line 1209, in fetchPage
FileNotFoundError: [Errno 2] No such file or directory: '<snip>'

I did open the program from the D drive but I dont have a <snip> folder on there. I couldn't get it to open since it would give me the error and by clicking "ok" it would close down the program. So I just manually created the following folder on my D drive and the program would then run.

<snip>

After opening the default download folder was the before mentioned <snip> so I changed that to the regular folder I normally use and tried to download a tease but it wouldn't work and it would give me the following error:

Traceback (most recent call last):
File "MiloDownloader.pyw", line 555, in <module>
File "DownloadLib.py", line 548, in get_tease_script
File "DownloadLib.py", line 676, in fix_Guideme_library_issue
KeyError: 'files'

I then tried it without changing the default download folder but it would still give me a error. I also tried it with changing or not changing the "Cache" folder on the Options tab but to no success.

I then decided to be a smart software tester and I opened the program again, changed the download default folder (and "Cache" folder) to my prefered folder and tried a different tease and that one worked. Tried a few more teases and they all worked.

Went back to the first tease I tried and that one still doesnt work so I'm guessing the error is specific to that tease (https://milovana.com/webteases/showtease.php?id=68913)

Well, apparently following my usual pattern I've managed to make a mess of things.

I inadvertently packaged a config.cfg file in the download that was pointing to folders on my computer, and referencing the tease I last tried testing. I've taken down the downloads and will repackage it and re-upload them, including fixes so I don't repeat that mistake in the future.

If you don't mind editing your message and removing references to that folder name I'd appreciate it.

And, you are apparently very lucky because the first tease you tried (68913) has uncovered a couple new bugs I hadn't found yet. I will fix those as well before I upload the new package.

Be back soon

PG

[PlayfulGuy]

The issues reported have been fixed, and a new version uploaded.

The download links in the first post of this thread have been updated.

Link to Virustotal details

2024-02-06 16_10_37 VirusTotal.jpg (124.66 KiB) Viewed 7578 times

[PlayfulGuy]

Thanks so much for all the time and effort you put into the downloader! I love the functionality of it and use it a lot to be able to play the teases offline. Very happy to have it working again.

You're very welcome by the way! Glad to know people use and enjoy it.
I was too busy feeling silly to acknowledge that earlier.

Went back to the first tease I tried and that one still doesnt work so I'm guessing the error is specific to that tease (https://milovana.com/webteases/showtease.php?id=68913)

It turns out this was caused by the forward slash in the tease title and hence filename and media folder. I've added code to fix that. It's kind of amazing I've never encountered that one before.

Since you've already tried downloading it and the downloader now remembers the title etc. you will need to edit the file name to change the slash to a dash (or whatever). If you update the filename then press tab it should automatically update the media folder for you. If not edit that too. It should then download successfully (it did for me anyway). You can leave the slash in the tease title, it's just the file name and media folder that need to be adjusted.

Enjoy

PG

[pagengifflarkanel]

That was a very quick fix! Thank you very much!

I re-tried the first tease again and it indeed works fine now, the folder and file name are adjusted with a "-" and not a"/".

Amazing work once again!

[whd]

Updated Feb 6/2024 to version 0.97, latest release, fixes download errors caused by recent Milovana updates, and more! See the Release Notes

Thank you for your effort to build this downloader, which I really like.

Despite your excellent answer about GuideMe and javascript description viewtopic.php?p=349324#p349324, I programmed the Teasefight Invitational Cup with EOS as an online Tease instead of offline for GuideMe. I hoped, that I could get both by using your download tool. And with EOS, there's a larger audience to give feedback and further ideas which helps to bring the tease forward.

Unfortunately, this didn't go well. I cannot download my own tease. https://milovana.com/webteases/showtease.php?id=69023


As you wrote

...most more advanced EOS teases will require some editing or adjustments.

Can you give me some tipps, where I have to start with, please? There goes nothing. It starts with black screen and even isn't able to load the very first page.
I reckon, that my extensive usage of * wildcards could give problems. I might be able to fix this, when the tease works to some extend. But if it doesn't start at all, I am lost.

Maybe, to know the original EOS tease code makes it easier to answer my question. Therefore I append also the json.

[Carnal1]

The issues reported have been fixed, and a new version uploaded.

The download links in the first post of this thread have been updated.

Link to Virustotal details
2024-02-06 16_10_37 VirusTotal.jpg

I just updated to your latest (.97) version and seems it still has the issue pagengifflarkanel describes in that it gives an error and shuts down unless the user remembers to set the download folder to something before attempting to download a tease. Once a download folder is set, all runs fine.
Thanks very much for this and all the rest you've shared with Milovana over the years. MiloDownloader has been a handy tool many times.

[PlayfulGuy]

The issues reported have been fixed, and a new version uploaded.

The download links in the first post of this thread have been updated.

Link to Virustotal details
2024-02-06 16_10_37 VirusTotal.jpg

I just updated to your latest (.97) version and seems it still has the issue pagengifflarkanel describes in that it gives an error and shuts down unless the user remembers to set the download folder to something before attempting to download a tease. Once a download folder is set, all runs fine.
Thanks very much for this and all the rest you've shared with Milovana over the years. MiloDownloader has been a handy tool many times.

Thanks for reporting this.

I think what's happening is that the downloader had a default of C:\temp for the download folder, so if that folder does not exist it might fail, but it does exist on my machine so it worked fine.

I thought I had it set so it defaults to the current working directory after a clean install, but I made a bunch of changes to how the download folder is handled in the last release so probably broke something. I'll post an update as time permits.

Thanks again,

PG

[PlayfulGuy]

Updated Feb 6/2024 to version 0.97, latest release, fixes download errors caused by recent Milovana updates, and more! See the Release Notes

Thank you for your effort to build this downloader, which I really like.

You're very welcome! I'm glad others find it useful too.

I cannot download my own tease. https://milovana.com/webteases/showtease.php?id=69023


...

Can you give me some tipps, where I have to start with, please? There goes nothing. It starts with black screen and even isn't able to load the very first page.
I reckon, that my extensive usage of * wildcards could give problems. I might be able to fix this, when the tease works to some extend. But if it doesn't start at all, I am lost.

Wildcards should not cause any issue. Support for those has been in the downloader since very early versions and is pretty solid.

Usually what you describe is caused by the downloader messing up the conversion of javascript code. The downloader can only convert relatively simple javascript, and even some javascript that seems rather basic gives it problems. It has particular trouble with code that uses objects to store data values.

I don't have time to do it today, but I will have a look at your tease over the next few days (hopefully tomorrow) and see what I can find.

In the meantime check the GuideMe log files for clues, and have a look at the initializeScriptVars() function in the downloaded file, and any other javascript that follows it. This is all the code that got converted from your original tease.

I'll get back to you soon,

PG

[PlayfulGuy]

Despite your excellent answer about GuideMe and javascript description viewtopic.php?p=349324#p349324, I programmed the Teasefight Invitational Cup with EOS as an online Tease instead of offline for GuideMe. I hoped, that I could get both by using your download tool. And with EOS, there's a larger audience to give feedback and further ideas which helps to bring the tease forward.

Unfortunately, this didn't go well. I cannot download my own tease. https://milovana.com/webteases/showtease.php?id=69023

So, I got your tease downloaded and spent a couple hours on it. As I expected there were several spots where the downloader broke javascript code when trying to convert it for GuideMe. I've attached a working copy of the xml file. Milovana doesn't allow xml attachements so just remove the ".zip" extension from the end. You may also need to update the MediaDirectory value if you used a different filename for your download.

I added notes at the beginning of the file describing what I found and how I fixed it. I'll summarize here, since others may find the information useful also, but there are more details in the attachment.

I can't guarantee that everything is working properly, but what I tested seemed to be working okay.

What I found

• The tease "init" code defined an object called game that included several properties and methods. GuideMe does not handle saving and restoring objects with methods (maybe the current version is better), but it turns out that the object was only used for it's random() function and none of the other properties or methods were referenced anywhere.
  
  I added function getRandom() based on the game.random() function and then replaced all calls to game.random() with calls to getRandom()
• The downloader corrupted a couple variable definitions in the init module.
  The original lines were

  Code: Select all

            var fighterload = "girlone,girltwo";
            var teaseload = "2fighttime,2fightspeed,2fightpattern";
  

  but the downloader screwed them up. I fixed those in this file but don't have time to figure out what happened and fix the downloader right now.
• The downloader also broke several functions.
  Several functions accept a "girllist" parameter, like getgirl(girllist) for example.
  The downloader added a getVariable() statement to the function that ended up redefining the passed list and broke the code. I commented out the extra statement and that resolved the issue.
• You should also check setting of "ForceStartPage". Maybe it should be false so you can close the tease and open it again later and continue from where you left off?
• In function initializeScriptvars() maybe set initialize_on_every_load to false? Probably some stuff should be reset every time, but others maybe should not. I added a couple comments in that function to show how you can manage a mix of stuff that needs to be reset everytime and stuff that does not.
• The init code defines many variables that are never actually used, like starttime, endtime, lengthmin and lengthmax for example. I assume the code was just copied from somewhere else and not cleaned up, so no big deal. There are too many so I didn't check them all, but leaving them won't hurt anything

I'll leave it to you for now, but please test it. Like I said, I can't guarantee that everything is working properly yet.

Feel free to ask questions here or in a PM if you run into further issues, but be patient. I try to check in here regularly, but sometimes I'm more regular than others

Good luck!

PG

[whd]

What I found

• The tease "init" .....
• The downloader corrupted a couple variable definitions in the init module....
• The downloader also broke several functions....
• You should also check setting of "ForceStartPage".....
• In function initializeScriptvars() maybe set ....
• The init code defines many variables that are never actually used, like starttime, endtime, lengthmin and lengthmax for example. I assume the code was just copied from somewhere else and not cleaned up, so no big deal. There are too many so I didn't check them all, but leaving them won't hurt anything

I'll leave it to you for now, but please test it. Like I said, I can't guarantee that everything is working properly yet.

Good luck!
PG

Wow, PlayfulGuy. You went all through. Thank you very much. It is working properly.
And you found out about every not cleaned dead code, which I startet with to implement something later or just took as starting code, as you guessed exactly right.
But with the downloaded and working tease, I am thinking over the issue, whether speed is good enough or not. My tease is written in a manner, which obviously makes it hard for the tease downloader. It is running now, but the delays seem to be a problem, since the fight system is very time sensitiv. EOS cannot handle indirect picture pointers and I used a nested workaround. XML could do it easily and hence, the nested EOS workaround makes the XML slower than it has to be.

As I said, I am gonna sleep over it. I made some improvements on the online tease as well and I have to decide, whether I want to clean all the stuff on the newer version, as you have done on version 1.1 or if I even want to write the tease as an offline tease. With all the thoughts while writing for EOS, it might be not so hard anymore to translate it to XML. Maybe I will also try this challenge.
What's hindering? My programming experience is very old (assembler, Pascal ) and little as well.
No knowledge of Java, HTML, XML and all the new stuff.

[PlayfulGuy]

I posted a few months back asking about being able to get descriptions and tags into the offline teases and displayed in GuideMe. As this wasn't something trivial that could just be added, I had a play with it myself, with the goal of creating a library of tease tags I could search offline.

Things went better than expected, and I was able to write a python script to scrape descriptions and tags from Milovana, then create new library txt files with more information. GuideMe appears to be limited to two lines of information, and only the first line wraps so I formatted the info within those constraints and updated the library. I was pleasantly surprised to be able to get it all working with relatively little trouble. I've attached a screenshot of the result. If anyone is interested in getting hold of the code I used feel free to pm me - I'm not an experienced coder so it's probably terrible code, but it worked for me.

That's very impressive!

Please send me a copy of your code so I can look it over and see what might be involved in adding this ability to the downloader.
The downloader is also written in python, and now that you've worked out the how of it, it might not be too difficult to incorporate it as an additional option in the downloader.

And a sample of one of your updated library text files would be good too.

Or maybe yours is a good standalone tool to complement GuideMe and the downloader...

Thanks,

PG