DIMMiN

Created at: May 7, 2026, 12:14 p.m. Updated at: May 7, 2026, 12:05 p.m.

2026-05-07-Thursday

Made sure that Catalixiahbot was operational for the subathon kick-off (it is up and running)
Identified potential start messages (either fcb39332-6d00-457c-b900-3532f97329f1 or 567c45c8-a7e0-48f0-8626-ac6578f99289) which we can switch to if we want to change the start time of the subathon.
Added an additional table display to the subathon sheet using XLOOKUP to show total watchtime and chat points broken apart by day for two weeks (requested by Catalixia)

Created at: May 7, 2026, noon Updated at: May 6, 2026, 1:01 p.m.

FINALLY Fixed the kernel level issue in my Homelab that prevented me from easily starting and stopping Docker containers thanks to this resource
Set up a CRON Job that restarts Catalixiahbot every week so it can continue to collect data
Figured out how to establish a long-term SSH key so I can transfer backup databases within my Homelab (between AM and MsNASty)
Set up another CRON Job that transfers the database to a folder in the NAS which handles versioning and database snapshots for me.
Found out that catalixia is starting her subathon on May 6, 2026 at 3 pm PST
Added a new feature to the Catalixiahbot that tracks the total number of chats daily
Added a new feature to the Catalixiahbot that tracks the total number of watchtime hours for a given day (however this is based on TwitchUserSessions and when they start, so they could be much higher than is reasonable)

Created at: May 5, 2026, noon Updated at: May 6, 2026, 7:30 a.m.

Decided to break apart the CupUp - An Analysis of Optimal Coffee blog post into a multi-part series to split up the writing component
Updated processing of variety, processing_system, and altitude_meters_lo / altitude_meters_hi
Set the maximum altitude to 2,600M during processing
Began working on validations for this final processing step

Created at: May 2, 2026, noon Updated at: May 2, 2026, 11:12 a.m.

Finished processing the farm_size_square_meters feature
Began updating the 2_clean_features.py script to finish numeric conversion of different fields

Created at: May 2, 2026, noon Updated at: April 30, 2026, 7:53 a.m.

Constructed a test in the Python pytest validator that identified 38 lots in the competition page where the link in the competition page does not actually match the lot URL (usually the Slug is changed in the competition page but redirects to the appropriate lot)
Integrated warnings (instead of test failures) into the validator to check for outlier auction scores
Identified a disconnect between listed lot size and the auction pricing results. For instance for Brazil Pulped Naturals 2000 the highest lot size is 220 30kg boxes (14,550.51 lbs), but the total price (just over $29k) and price per lb ($1.02 / lb) suggest a total size of 29,102 lbs.

Created at: April 29, 2026, noon Updated at: April 30, 2026, 4:27 a.m.

Extracted all numeric values from the competition pages, including price_per_unit, total_lot_value
Extracted total lot size (in lbs) using total_lot_value and price_per_unit
Completed update of 1_incorporate_competition_data.py

Created at: April 25, 2026, noon Updated at: April 25, 2026, 11:10 a.m.

Re-worked 1_incorporate_competition_data.py to extract relevant data row-by-row for each table instead of trying to designate a price or auction table. This resulted in much higher fidelity data, however there are still edge cases resulting in about ~10% of the data not including a field for the price_per_unit feature.
Began converting competition columns to numeric features (prices, sizes in kg, etc.)
Visualized the distribution of the target variable (price_per_unit), looks like it follows an almost perfect Log Normal Distribution

Created at: April 24, 2026, noon Updated at: April 24, 2026, 11 a.m.

Began updating the second script in the pipeline, 1_incorporate_competition_data.py by switching away from a pandas processing job to using raw Python Dictionarys

Created at: April 24, 2026, noon Updated at: April 24, 2026, 2:50 a.m.

Started building out a validator using Python pytest to validate each of the steps involved in processing the cup of excellence data
Caught a bug where some files were incorrectly named, re-collected these files and fixed them
Explicitly handled cases where competition pages did not have links to individual lots

Created at: April 12, 2026, noon Updated at: April 22, 2026, 11:32 p.m.

Built and deployed an interactive A/B testing demo here to help secure a potential client contract.