🔍 MiloDB: Finding Old Teases

[FrozenWolf]

Table of Contents
1. Overview
2. Downloads
3. Examples of Use
4. FAQ
5. Change History
6. Launch Configuration

1. Overview
MiloDB is an offline application and database that provides enhanced search capabilities to find Milovana teases. It supports EOS, Flash, and Regular tease formats.

Searches may look for text in the following fields:

• Title
• Summary
• Text content
• Tags

Searches may also compare against the following fields:

• Date published
• Tease ID
• Author name
• Author ID
• Type (eos/nyx/reg/aud)

All HTML tags, Javascript, formatting, nagivation logic, page identifiers, etc. are stripped from the text fields leaving only the raw text content.

1.1. Interface
The tool runs in an interactive terminal. Commands are typed at the prompt to perform searches, preview results, change format, etc. This type of interface isn't for everyone, but you'll get the hang of it with a bit of practice.

The "help" command can be used to provide general information on the list of commands available or detailed information on a specific command. The TAB key can be used to complete commands and provide a list of suggestions where appropriate.

See "3. Examples of Use" for more details.

1.2. Platform Support
The tool is written in Python and has been built into a single executable with pyinstaller for Windows and Linux.

1.3. Limitations

• Tags are obtained from the Milovana webpages, and hence are likely to be a subset of the actual tags that the author of the tease registered.
• The database will be some months behind the teases available from the Milovana website. This provides some time for the content and rating of the teases to settle before they are captured.
• Teases that are changed after they've been captured won't be updated in the database because there is no easy way to determine that they've been changed.
• Teases that have been removed after they've been captured may still reside in the database because there is no easy way to determine that they've been removed.
• Chinese/Japanese/Korean etc. characters may be shown as square boxes in the Windows console.

1.4. Thanks

• Many thanks to PlayfulGuy for invaluable alpha testing and excellent suggestions in many areas.
• Many thanks to Trusfrated for invaluable beta testing and superb ideas to make the tool a lot more user friendly.

[FrozenWolf]

2. Downloads
Most users will just want the binaries. The source is available for the curious, but is not required to run the tool.

2.1. Self Updating
From version 3.0.0, there is a built in '

update

' command that should allow you to easily update to the latest version. Note that it only works with the binary distribution, not the source code.

2.2. Binaries
milodb-2024-12-31-3.8.0.zip
Extract the zip file and you should get the following directory structure:

Code: Select all

milodb/
├── changelog.md         <- History of changes
├── database.mdb         <- Database of teases
├── default-light.css    <- Style sheet used for 'browse' and 'browseall' commands (default)
├── default-dark.css     <- Style sheet used for 'browse' and 'browseall' commands
├── milodb               <- Linux executable
├── milodb.exe           <- Windows executable
└── milodb.ver           <- Local manifest

Windows users should run (e.g. double-click) '

milodb.exe

'.

2.3. Source
milodb-2024-12-31-3.8.0-src.zip
Extract the zip file and you should get the following directory structure:

Code: Select all

milodb-src/
├── changelog.md         <- History of changes
├── default-light.css    <- Style sheet used for 'browse' and 'browseall' commands (default)
├── default-dark.css     <- Style sheet used for 'browse' and 'browseall' commands
├── LICENSE.txt          <- License file
├── milodb_client/       <- Client source code
├── milodb_client_test/  <- Client unit tests
├── milodb_common/       <- Common (client and admin shared) source code
├── milodb_common_test/  <- Common unit tests
├── run.py               <- Start script
├── test.py              <- Run unit tests
└── tools/               <- Various utilities

Note that the source zip does not include the database which can be obtained from the binaries.

You can install the necessary requirements with:

     pip -r requirements-file

where

requirements-file

is one of these:

• tools/requirements-run.txt

  
  If you just want to run the code with '

  run.py

  '

• tools/requirements-build.txt

  
  If you just want to build the executable with pyinstaller manually.
  Note: This includes

  requirements-run.txt

  .
  Note: If you use the '

  tools/build-bin.*

  ' scripts, they use '

  requirements-build.txt

  ' to acquire those packages into a virtual environment.

• tools/requirements-develop.txt

  
  If you just want to run the tests with '

  test.py

  '
  Note: This includes

  requirements-build.txt

  .
  This comprises the full set of tools that I use for development.

2.3. Old Builds

Spoiler: show

Binaries
Sources

[FrozenWolf]

3. Examples of Use
This is a brief summary of a few ways that you might use the tool.

3.1. Starting Up

Milovana Database 1.0, 2022-03-31
Saved config file 'config.json'
Load database
  Read     : 0.007s
  Unpack   : 1.043s
  Parse    : 0.352s
  Assemble : 0.143s
Load completed: 1.546s
7,451 teases loaded ranging from 2006-08-15 to 2021-12-31

Enter command (or "help"):
>

The line "

Saved config file 'config.json'

" indicates that a default config file has been created because one was not found. Subsequent executions will read the existing config file.

3.2. Searching for 'mystical'
Search all text content of all teases for the word 'mystical':

> query text contains mystical

  1:  #18461  nyx  4.4     2012-10-14  Devious : [1885, @17204]
  2:  #30545  nyx  4.4  w  2015-07-17  The Mystical Realm of Myrewood Isle - Version 1.2 : [r0nin47, @4316]
  3:  #34481  nyx  3.9  n  2017-04-14  Chastity 4: Rebellion against the Goddess : [Shattered, @29180]
  4:  #37873  nyx  4.5  n  2018-08-23  The Curse : [MisterFlames, @101953]
  5:  #40047  eos  4.5  n  2019-03-05  Three Ideals Fantasy Game : [Shymaroon, @103415]
  6:  #40253  eos  4.7  n  2020-05-01  Chastity 7α: Devote a month to Xia : [Shattered, @29180]
  7:  #40411  nyx  4.0     2019-05-17  The Dungeons of Myrewood Isle : [r0nin47, @4316]
  8:  #44614  reg  4.5     2020-03-03  SLUTRIX FUCKLUTION - PART3 : [bvb09, @29476]
  9:  #45128  eos  4.0  w  2020-04-11  Enter The Pony Maze : [Morexis, @104459]
 10:  #45571  eos  4.1  w  2020-05-26  Lost in Lewd Labyrinth : [Morexis, @104459]
 11:  #46529  eos  4.3  w  2020-08-11  League of Teasing : [LongjumpingFudge, @107434]
 12:  #53235  eos  4.8  w  2021-10-27  Trials of the Succubi v1.7 : [indyc, @110114]

The '

contains

' verb used in the query searches for plain text.

Show details of the 9th match in the list to see where the word appears in the text:

> show 9

Index         9
Tease    #45128   Title  'Enter The Pony Maze'
Author  @104459   Name   'Morexis'
Type        eos   Date   2020-04-11
Rating      4.0   Tags   totm, cbt, mlp, female-top, anal, maze
Tease   https://milovana.com/webteases/showtease.php?id=45128
Author  memberlist.php?mode=viewprofile&u=104459
Summary
You wake up in the middle of a magical maze full of horny ponies and your only way out is a locked gate. Left with no alternatives, you embark on a journey to explore the maze and retrieve all keys...
Matching text
----------------------------------------
…ow, we're in the middle of a large and mystical maze.
You can move freely from room to…
----------------------------------------
They're mystical objects and only those who know the ma…
----------------------------------------
…st be exhausted from running around my mystical maze.
----------------------------------------

Open the tease in the default web browser:

> open 9

Opening 'https://milovana.com/webteases/showtease.php?id=45128': Enter The Pony Maze

3.3. Searching for '8-ball'
Search all text content of all teases for the word '8-ball' or 'eight-ball' or '8 ball' or 'eight ball' or '8ball' or 'eightball':

> query text matches "(8|eight)[ \-]?ball"

  1:    #200  reg  3.6  2006-10-12  8-Ball : [slavechylde, @284]
  2:    #997  reg  3.6  2007-08-25  Poolhall Jerkies : [HogtiedCowboy, @1415]
  3:   #2760  nyx  3.7  2008-06-29  Jessica's Game of Cards : [Evals, @1311]
  4:  #13914  reg  3.9  2011-10-29  Nipple Edge : [footstool, @1053]
  5:  #14585  nyx  2.1  2011-12-12  Playing card hunt. : [diddums, @1015]
  6:  #37490  nyx  2.8  2018-04-05  Mistress Karen´s Game : [Mistress Karen, @101809]
  7:  #42810  eos  4.5  2019-12-07  Sister Tease : [BopIT, @102906]
  8:  #49411  eos  4.4  2021-03-06  Riley's mix game : [, @0]
  9:  #53313  eos  4.8  2021-12-05  Maze-turbation Marathon : [Animeat, @110995]

The '

matches

' verb used in the query searches for a regular expression. This is an extremely powerful (although sometimes tricky) method of searching. For some guidance on the use of regular expressions, see https://pythex.org/.

Create a webpage summarising the results and open it in the default web browser.

> browse

Opening '/tmp/tmpcxf4ezev.html'

3.4. Searching for multiple fields
Using boolean operators and parentheses, multiple fields can be searched for at one:

> query date >= 2010-01-01 and (type is nyx or type is eos) and title contains goddess and rating > 4 and tag contains totm and text matches "chastity (belt|cage)"

  1:  #33609  nyx  4.3  2016-10-31  Chastity 3: Chastity for the Goddess : [Shattered, @29180]
  2:  #34029  nyx  4.2  2016-10-19  GODDESS WORSHIP - First days of servitude : [PrincessJulia, @41437]

Copy the URL of the author of the 2nd tease in the list to the clipboard.

> copy urlauthor 2

https://milovana.com/forum/memberlist.php?mode=viewprofile&u=41437

[FrozenWolf]

4. FAQ

Will there be a graphical interface?

I'm working on a graphical interface in the background, but it's a huge amount of work and won't likely appear for some time.

Can I change the colours in the terminal?

Yes, the '

config.json

' file has a section "

format/ansi/colour

" that contains ANSI colour codes used when the format is set to 'ansi'. See https://en.wikipedia.org/wiki/ANSI_escape_code#Colors for some guidance.
You can use the built in

config

command to change the colours at runtime, for example:

  > config set format/ansi/colour/matchingText 1;41

Can I change the colours in the BBCode format?

Yes, the '

config.json

' file has a section "

format/bbcode/colour

" that contains BBCode colour codes used when the format is set to 'bbcode'. This is typically represented as a hexadecimal octet triplet in the form '#rrggbb' where 'rr', 'gg', and 'bb' represent red, green, and blue values respectively. See https://www.w3schools.com/colors/colors_picker.asp for some guidance.
You can use the built in

config

command to change the colours at runtime, for example:

  > config set format/bbcode/colour/matchingText #80FF00

Can I change the colours/styles in the generated HTML web pages?

Yes, the '

config.json

' file has a section "

format/html/cssFilePath

" that refers to the active stylesheet that is injected into the generated web pages. You can change this to one of the default stylesheets, or use one of your own.
You can use the built in

config

command to change the stylesheet at runtime, for example:

  > config set format/html/cssFilePath default-dark.css

Why do logogram fonts such as hanzin, kanji, and hanja appear as square boxes?

Windows console support for unicode character sets isn't particularly good. There's not much the tool can do about that, but using the 'browse' command to view the results in a web browser is a workaround.

What do 'w' and 'n' represent in the results list?

'w' represents a 'tease-of-the-month winner' identified by the tag 'totm'.
'n' represents a 'tease-of-the-month nominee' identified by the tag 'totm-nominee'.

What do the gold and silver icons represent on the generated web pages?

Gold represents a 'tease-of-the-month winner' identified by the tag 'totm'.
Silver represents a 'tease-of-the-month nominee' identified by the tag 'totm-nominee'.

Why is the database some months out of date?

After any tease is released, there is an opportunity for the author to update it and users to play and rate it. It seems sensible to allow a settling period for that to occur before the tease is captured. Additionally, this tool is intended to find teases that have fallen off the first few pages of the webteases list.

How often is the database going to be updated?

My current intention is to update it approximately every month.

Does this tool put a lot of stress on the Milovana server?

No, the tool runs completely offline. The only time the Milovana website is accessed is when your web browser retrieves thumbnails for the web pages generated by the 'browse' and 'browseall' commands.

How is the database constructed?

The milovana website is scraped (by me) to retrieve the metadata from the webteases list, and the actual tease content. This forms a delta in a master-database. All deltas are merged together, reformatted, heavily compressed, saved to 'database.mdb' and packaged along with the application for users to download.

Does updating the database put a lot of stress on the Milovana server?

No. When the data is retrieved from the Milovana website, it's only fetching the HTML part of a few pages of webteases, and downloading the script component of a months worth of teases. It doesn't fetch any media (i.e. images and sounds) and therefore the amount of data retrieved is typically much less than just playing a single tease.

Can I use private browsing for the 'browse', 'browseall', 'open', and 'openauthor' commands?

Yes, the '

config.json

' file has sections "

commands/browse

" and "

commands/open

" to configure how the browser is launched. See section 6. Launch Configuration for more information.

[FrozenWolf]

5. Change History

3.8.0, 2024-12-31

• Fixed: 'unknown' author ID shown in browse summary had wrong colour in default-dark.css
• Changed: Upgraded to Python 3.12
• Changed: Fortified update fetches by handling incomplete reads and URI redirections
• Database: Updated to 2024-09-30

3.7.0, 2024-11-30

• Fixed: Matched fields 'status' and 'authorStatus' were associated with the wrong tease field
• Added: Dark themed CSS for 'browse' and 'browseall' commands
• Changed: Removed CSS colour configuration from config file and moved into CSS file
• Database: Updated to 2024-08-29

3.6.0, 2024-10-31

• Changed: Database format updated
• Database: Updated to 2024-07-30

3.5.0, 2024-09-30

• Added: Automatic migration to newer configuration versions
• Changed: Split launch options into platform specific groups
• Database: Updated to 2024-06-30

3.4.0, 2024-08-31

• Added: 'config set' subcommand to set entries at runtime
• Added: 'config edit' subcommand to edit entries at runtime
• Added: 'config reset' subcommand to reset entries at runtime
• Database: Sorted tags alphabetically
• Database: Replaced ambiguous punctuation with standard punctuation
• Database: Updated to 2024-05-31

3.3.0, 2024-07-31

• Added: 'about' command to show system info and forum links
• Added: 'config purge' subcommand to purge redundant entries from the configuration file
• Changed: Replaced pip requirements-{windows,linux}.txt with requirements-{run,build,develop}.txt
• Database: Replaced non-breaking spaces with regular spaces in tease text
• Database: Updated to 2024-04-30

3.2.0, 2024-06-30

• Fixed: 'query' command operator 'not' now has higher precedence as was originally intended
• Fixed: Unit test failure on Windows due to inability to delete read-only files
• Added: 'config' command to view config entries at runtime
• Changed: 'update' command will prompt and wait for user response if there is any error during the update
• Database: Updated to 2024-03-30

3.1.0, 2024-05-31

• Fixed: Help text of 'update' command had rogue linefeed and full stop
• Changed: 'query' command can now match against partial dates
• Changed: 'update' command re-uses remote connections
• Changed: 'update' command verifies integrity of downloaded assets
• Database: Updated to 2024-02-28

3.0.0, 2024-04-30

• Added: 'update' command to perform self updates
• Database: Updated to 2024-01-30

2.5.0, 2024-03-31

• Database: Updated to 2023-12-31

2.4.0, 2024-02-29

• Changed: Upgraded PyInstaller to 6.4.0
• Database: Updated to 2023-11-30

2.3.0, 2024-01-31

• Database: Updated to 2023-10-31

2023 Builds

Spoiler: show

2.2.0, 2023-12-31

• Database: Updated to 2023-09-30

2.1.0, 2023-11-30

• Removed: 'update' command to perform self updates
• Database: Updated to 2023-08-31

2.0.0, 2023-10-31

• Added: 'update' command to perform self updates
• Database: Updated to 2023-07-31

1.18, 2023-09-30

• Fixed: Username of @10570 in #8668 renamed 'paswis' -> 'mangoman'
• Added: Clicking on tease titles in browse pages will jump between list and summary
• Database: Updated to 2023-06-30

1.17, 2023-08-31

• Added: Known authors who have left are now prefixed with '[GONE]' with their name and authorId still present
• Added: 'query' command can query against absent authors with 'authorStatus' field
• Changed: 'url' and 'urlauthor' commands can now output more than one link at a time
• Database: Updated to 2023-05-26

1.16, 2023-07-30

• Database: Updated to 2023-04-29

1.15, 2023-06-30

• Fixed: NYX teases with unquoted text fields were not being captured
• Fixed: Config settings were always using default values
• Added: Page references can now be shown alongside text paragraphs
• Added: 'pagerefs' command to toggle the visibility of the page references
• Added: Configuration for launching the web browser with 'browse', 'browseall', 'open', and 'openauthor' commands
• Database: Updated to 2023-03-31

1.14, 2023-05-31

• Fixed: Line breaks in NYX text are now preserved
• Added: Contents of known deleted teases are now included in the database
• Added: The title of a deleted tease is prefixed with '[DELETED]'
• Added: 'query' command can query against deleted teases with 'status' field
• Changed: Replace webpage favicon with inline SVG
• Database: Updated to 2023-02-27

1.13, 2023-04-30

• Added: Index of the match is now show on webpages
• Changed: Webpage TOTM icons replaced with inline SVG
• Changed: Reduced executable size by excluding modules and binaries
• Database: Updated to 2023-01-31

1.12, 2023-03-31

• Fixed: Database had missing content for teases #39182, #42325, #43250, #44945, #46853, and #47882
• Fixed: Some NYX teases had missing line breaks due to malformed HTML
• Fixed: Exception when running test.py when milodb_admin_tests package does not exist
• Added: License statement in each source file
• Database: Updated to 2022-12-31

1.11, 2023-02-28

• Database: Updated to 2022-11-30

1.10, 2023-01-31

• Fixed: 'summary' command no longer shows author URLs when author is unknown
• Fixed: 'stats' command now shows '[unknown author]' instead of '[, @0]'
• Added: 'query' command can now query against tease-of-the-month field
• Changed: List alignment no longer shifts to the right when 99 < hits < 1000
• Database: Updated to 2022-10-31

2022 Builds

Spoiler: show

1.9, 2022-12-31

• Fixed: 'var delete/get' no longer ignores redundant arguments
• Fixed: Character index of an unterminated string now refers to the starting quote
• Fixed: 'browse' and 'browseall' no longer fail silently if the CSS file is not found
• Fixed: TAB completion was misleading when input included a fault in variable expansion
• Fixed: Variable expansion is now disabled for of all subcommands of 'var' command
• Fixed: 'copy' command now strips trailing end of line
• Fixed: Manually patched author names of '[email protected]' caused by Cloudfare obfuscation
• Added: Errors on input now display the input after variable expansion to give better context
• Added: 'var edit' subcommand
• Added: 'var copy' subcommand
• Added: 'var rename' subcommand
• Database: Updated to 2022-09-30

1.8, 2022-11-30

• Added: 'var' command to support inline input variable expansion
• Database: Updated to 2022-08-30

1.7.1, 2022-11-02

• Fixed: Author name was corrupted in summaries in ANSI format

1.7, 2022-10-31

• Fixed: Removed redundant '/' in '<meta.../>' in generated HTML
• Fixed: URLs are no longer generated for unknown authors
• Added: Sorting by number of hits
• Added: Hits statistics shown in lists and summaries
• Changed: Help of 'sort' command includes information on sort keys
• Changed: Placeholder shown when an author's name/id is missing
• Changed: Improved consistency of error messages on output
• Database: Updated to 2022-07-31

1.6, 2022-09-30

• Fixed: Text highlighting now works properly when highlight spans line-endings
• Fixed: Unhandled exception when 'copy' command couldn't access clipboard
• Fixed: Empty directories from distribution source zip file are now pruned
• Database: Updated to 2022-06-30

1.5, 2022-08-31

• Changed: Improved performance of 'show', 'showall', 'browse', 'browseall'
• Database: Updated to 2022-05-31

1.4, 2022-07-31

• Database: Updated to 2022-04-30

1.3, 2022-06-30

• Fixed: Help examples for 'show' command used the wrong command
• Added: 'showall' command to complement the 'browseall' equivalent
• Database: Updated to 2022-03-31

1.2, 2022-05-31

• Fixed: Ellipsis in abbreviated text was being placed incorrectly in some circumstances
• Fixed: Ellipsis was still shown with 'show' command even when disabled on ANSI format
• Added: 'copy' command to transfer output of the following command to the clipboard
• Added: Extended 'stats' to include 'images' parameter
• Added: Protection against excessive memory consumption by constraining total match count
• Database: Updated to 2022-02-28

1.1, 2022-04-30

• Fixed: Query text now matches against capital letters
• Added: 'stats' command for silly statistics of the database
• Database: Updated to 2022-01-31

1.0, 2022-03-31

• Database: Updated to 2021-12-31

Beta Builds

Spoiler: show

0.9, 2022-03-30, beta

• Added: Indication of TOTM nominees and winners are now shown in the list view as 'w' or 'n'
• Changed: Removed text of '1st' and '2nd' from TOTM icons in generated webpages

0.8, 2022-03-22, beta

• Fixed: Tags for TOTM nominees and winners are now patched into the database
• Fixed: Errors from missing or invalid 'type is' parameters now show a list of options
• Added: Sort by TOTM is now possible with 'totm' sort key
• Added: Help for query now includes example for using 'not' operator
• Added: Webpages produced by 'browse' and 'browseall' have an icon and a better title
• Added: Webpages produced by 'browse' and 'browseall' show a TOTM icon for relevant teases
• Added: Queries for 'type is' has TAB support for suggestion and completion
• Added: Windows - Executable now has an icon
• Changed: Removed 'tags' as a queryable field
• Changed: Improved formatting of help output
• Changed: Displayed list of expected possible inputs on error is now sorted

0.7, 2022-03-17, beta

• Fixed: Sorting results by rating is now consistent
• Fixed: Teases with no rating showed a rating of "0" instead of "0.0"
• Fixed: Rogue characters in titles and summaries are now stripped
• Fixed: Tags are now shown as a list of items
• Added: Query field 'date' supports verbs '>', '<', '>=', '<=', '='
• Added: Tags can now be searched as separate items
• Added: Field 'tag' is now a synonym for 'tags'

0.6, 2022-03-13, beta

• Added: Query verb 'matches' is for regular expression matches
• Added: 'browse' and 'browseall' commands include thumbnails for each tease
• Changed: Query fields 'rating', 'authorId', 'teaseId' supports verbs '>', '<', '>=', '<=', '='
• Changed: Query verbs 'is' and 'contains' are now plain text matches
• Changed: Query verbs are now appropriate to each field

0.5, 2022-03-07, beta

• Fixed: Alignment of the output of the 'show' command when authorIds are more than 5 digits
• Changed: Rating is now shown as a value instead of a percentage
• Changed: Config file is no longer distributed with the release
• Changed: Config file is automatically generated with defaults if missing
• Changed: Config file is verified for missing, redundant, and invalid entries
• Changed: 'browse' and 'browseall' now use the CSS file specified in the config file
• Changed: Default CSS file "default.css" is now distributed with the release
• Changed: CSS files may contain keywords that are replaced with config entries
• Changed: Trimmed the output of the BBCode formatting to remove redundant [pre] tags

Alpha Builds

Spoiler: show

0.4, 2022-02-19, alpha

• Changed: Replaced pack files with single database file

0.3, 2022-02-14, alpha

• Added: 'url' and 'urlauthor' commands
• Added: 'openauthor' command
• Added: Run list of commands in autoexec.txt at startup if it exists
• Changed: Command 'list', 'show', and 'summary' now show indices for matches
• Changed: Commands that used to accept a TeaseId now accept a RefId which can be a MatchIndex, #TeaseId, or @AuthorId
• Changed: Output formats ansi, plain, and bbcode now prefix TeaseId with '#' and AuthorId with '@'
• Changed: Improved help output

0.2, 2022-01-15, alpha

• Added: Readme file
• Added: Changelog file
• Fixed: Windows - Crash on browse and browseall with unicode characters
• Fixed: Windows - ANSI colour codes showing as raw characters in terminal
• Fixed: Windows - Single file executable has rogue dependencies
• Changed: Extended help for query command

0.1, 2022-01-11, alpha

• Added: 'help' command
• Changed: Load database packs in parallel

0.0, 2022-01-03, alpha

• Initial release

[FrozenWolf]

MiloDB 1.1, 2022-04-30

• Fixed: Query text now matches against capital letters
  This was a silly regression mistake that my tests didn't catch.
• Added: 'stats' command for silly statistics of the database
  Some authors have written entire novels. Very impressive!
  Try the command '

  stats content

  ' to see what I mean.
• Database: Updated to 2022-01-31

I began working on a GUI as an addition to/replacement for the console interface. Currently I've had no feedback as to whether anyone has used (or even downloaded) MiloDB 1.0, so I'll put the GUI development on hold for now.

[Trusfrated]

Thanks for the update!

The new Stats addition is pretty cool. Very eye-opening.

This is a great little utility. It's not like I use it everyday, but it's indispensable when I need to find something buried in the depths of this site. The stock search system is severely lacking and this makes up for it.

[FrozenWolf]

Thanks for the update!

The new Stats addition is pretty cool. Very eye-opening.

This is a great little utility. It's not like I use it everyday, but it's indispensable when I need to find something buried in the depths of this site. The stock search system is severely lacking and this makes up for it.

Thanks Trusfrated, at least I know it's been used at least once . Let me know if you have any further suggestions for improvements, I'm always open to ideas.

[oculus]

Great tool! Even without a GUI it works like a charm ;)
Really missed this kind of search on the page itself.

What would be great, if i am able to update the database by itsself. Is there anyway to do it by Script?

[FrozenWolf]

Great tool! Even without a GUI it works like a charm ;)
Really missed this kind of search on the page itself.

Thanks, that's great to hear!

What would be great, if i am able to update the database by itsself. Is there anyway to do it by Script?

This was a quandary for me during the early alpha builds. Generating the database is a tricky job that involves scraping the website, parsing the tease, stripping any formatting, encoding, validating, and finally compressing. I have an admin tool that does most of the heavy lifting for me, but there are still a few manual steps.

There were a number of options that I considered specifically for the application:

1. Include an 'update' command to pull the latest pre-built database (or delta patch)
   With this method, the release binary wouldn't need to include the database, although the merging of delta patches is quite involved.
2. Include an 'update' command to scrape the latest from the Milovana website
   This could potentially put a huge strain on the Milovana website, so I rejected this idea early on. Instead I do a one-off scrape with a separate admin tool so that there is minimum impact on the website. That does mean of course that the database will always be out of date, but I think that's the most fair I can be.
3. Deliver the database and application together
   This puts all of the strain on some other server, in this case Mega, and the total binary file size is probably rather trivial to Mega. The format of the database is still being updated/tweaked to better support the internal workings of the application, and at the moment, the version of the application and version of the database are closely tied together.

There were a few other options, but those were the most relevant that I considered, and eventually opted for option #3 for its simplicity and negligible load on the Milovana website. This could change in the future, and would likely transition to option #1, in which case the application would have to be capable of updating itself to deal with any database format changes, so I think it would be more of an 'update all' command.

If you have any other ideas, it's always nice to hear more options!

[oculus]

Just some rough thinks for your consideration, since I don't know much about your admin script and the database itsself :)

Firstly i would recommend hosting the source on e.g. Github.

This would probable help you with Option 1 a lot and should be quite easy to maintain.
Also it helps you for Release Notes, Updating the Database, source diffs and maybe someone will help you developing ;)
It is possible to do a git-commit via Console, therefore it could be included in your admin-script. With the nice sideeffect for your users, that you can implement a auto notification and download on start of the app :)

Since your app is mainly written in python (as far as i know, please correct me if i'm wrong) you could also provide the runtime for python and patch the script files via git-diff from GitHub. Would save you the hasle of providing a binary all the time. Just need one binary, with contains your Patch-System and loads anything from GitHub (Spicy for Tease AIJ is a gread example how it could work)

What i would do for option 2 is the following:
1. Check page 1 for latest Tease, if this one is already inside DB -> Stop updating
If this precheck is done, the load for page only comes if there are teases to update. In any other cases it only produces the same load as if I'm opening the tease page.
To further give you some advices i need to understand how your database is constructed and how your admin scripts work :)

If you're willing to share some insights i'll try to help you as best as i can.

[FrozenWolf]

Firstly i would recommend hosting the source on e.g. Github.

This would probable help you with Option 1 a lot and should be quite easy to maintain.
Also it helps you for Release Notes, Updating the Database, source diffs and maybe someone will help you developing ;)
It is possible to do a git-commit via Console, therefore it could be included in your admin-script. With the nice sideeffect for your users, that you can implement a auto notification and download on start of the app :)

I already keep all of the source in a local git repo, but it's overly big for github at the moment because of its history; luckily I don't need any help with the development at the moment, but thanks for the offer!

Since your app is mainly written in python (as far as i know, please correct me if i'm wrong) you could also provide the runtime for python and patch the script files via git-diff from GitHub. Would save you the hasle of providing a binary all the time. Just need one binary, with contains your Patch-System and loads anything from GitHub (Spicy for Tease AIJ is a gread example how it could work)

Yes, it is almost entirely Python. You can have a look at the source snapshot available from here: viewtopic.php?p=323738#p323738.

Providing the binary is no problem at all, and there wouldn't be much of an advantage to hosting it on github at the moment. The admin tool does most of the heavy lifting. TAJ has its own complications due to the dependency on the Java runtime and external runtime libraries. I wanted to try and keep the whole tool offline to keep it simple.

My understanding from reading other community posts is that a single download and 'double-click-to-run' approach is preferred with as little complication as possible. That's why I'm keeping the distribution as simple as possible, at least for now. There are certainly many opportunities for alternative hosting and distribution techniques, so I'll keep this one in mind too.

What i would do for option 2 is the following:
1. Check page 1 for latest Tease, if this one is already inside DB -> Stop updating
If this precheck is done, the load for page only comes if there are teases to update. In any other cases it only produces the same load as if I'm opening the tease page.

This is actually what I'm trying to avoid; some users have already expressed concern of the potential hit to the Milovana website, especially as I don't expect users to use the tool very often, and performing an update after a month or two of new teases by multiple users would put unnecessary pressure on the website.

The primary purpose of the tool is to help find text within older teases that have fallen off the first few pages of the Webteases. Newer teases tend to be updated periodically and their rating bounces around all over the place; that's why I'm using the three month latency on the database contents. So it's not intended to be a mirror of the Milovana teases, or a replacement for the existing search facility; it's more of an archive tool really.

I hope that makes some sense, I think I've written too much again.

[FrozenWolf]

MiloDB 1.2, 2022-05-31

• Fixed: Ellipsis in abbreviated text was being placed incorrectly in some circumstances
• Fixed: Ellipsis was still shown with 'show' command even when disabled on ANSI format
• Added: 'copy' command to transfer output of the following command to the clipboard
• Added: Extended 'stats' to include 'images' parameter
• Added: Protection against excessive memory consumption by constraining total match count
• Database: Updated to 2022-02-28

'copy' command

Examples:

Code: Select all

> copy list
> copy show
> copy url 8

Note: The 'url' and 'urlauthor' commands no longer copy to the clipboard by default and their respective output to the terminal is now simplified to just the URL text without embellishments.

'stats images' command

The new 'stats images' command (suggested by Trusfrated and indyc) would have shown the following:

Code: Select all

> stats images

Top count of unique images by tease:
     10,317   #53235  Trials of the Succubi v1.7 : [indyc, @110114]
      4,062   #53783  Animation Quality Test with Blake Blossom [Demo] : [fapnip, @106621]
      3,795   #50990  Miss Evans (Repost) : [, @0]
      2,613   #35401  Will you save America ? : [lovecraft, @46155]
      ...

indyc, that's incredible!

I then discovered that '#53235, Trials of the Succubi v1.7, 2021-10-27' isn't there any more so I've removed it, but it has been superseded by '#56177, Trials of the Succubi, 2022-04-13' , although that doesn't actually appear in the current database because it's still quite new . It'll be in there eventually!

Protection against excessive memory consumption

If you perform a query that's likely to match against an enormous number of text items, it will be abandoned after reaching 10 million matches.
For example, if you search for all vowels (no idea why you'd want to, but hey why not?!):

Code: Select all

> query text matches [aeiou]

  1:   #6684  reg  3.3     2017-07-06  Pantyhose Dreams : [pha, @8971]
  2:  #12336  nyx  3.2     2016-06-24  Anal play : [shdw_fghtr, @16619]
  3:  #13201  nyx  3.5     2015-09-21  Fick meinen Dildo : [Blacky4096, @11068]
...
2407:  #50629  eos  4.3     2021-05-20  Exchange : [fibonacci, @693]
2408:  #50667  reg  3.5     2021-05-20  Domme PlayDate cum eating instructions : [ADLove, @110595]
2409:  #50714  eos  4.2     2021-05-24  Cum for this emerald eyed goddess : [megaman, @110384]

Warning: Limit of 10,000,000 matching items was reached
   Note: Search stopped with 9,998,555 matching items
   Note: Searched 2,418 of 7,517 teases (32.2%)

[PlayfulGuy]

Hi frozenwolf!

I had occasion to use this today. I was trying to remember what tease gave a rather specific instruction and with this tool I was able to find it in a matter of seconds.

Like trusfrated said in a previous comment, I may not use it every day, but it's indispensable when you need it.

Your instructions in the thread here were also invaluable. Nothing like good instructions to go with a great tool!
It was great to be able to do a "copy url" command, flip over to my downloader and paste the url, and a few seconds after finding the tease I was looking for it's downloading!

Thanks for all your hard work on this.

PG

[FrozenWolf]

Hi PG,

I had occasion to use this today. I was trying to remember what tease gave a rather specific instruction and with this tool I was able to find it in a matter of seconds.

Like trusfrated said in a previous comment, I may not use it every day, but it's indispensable when you need it.

Great, that's what I like to hear. If it gets the occasional use, that's good enough for me to keep it alive.

It was great to be able to do a "copy url" command, flip over to my downloader and paste the url, and a few seconds after finding the tease I was looking for it's downloading!

It's great that it works in tandem with your downloader (which is superb by the way). You get the credit for suggesting the 'url' command in the first place!