znc2tmsr_etc.kv.vpatch

August 18th, 2019

vpatch: http://lobbesblog.com/src/logotron/znc2tmsr_etc.kv.vpatch
sig: http://lobbesblog.com/src/logotron/znc2tmsr_etc.kv.vpatch.lobbes.sig


Author's notes:

Results from testing

After pressing the logotron and applying a few fixes to eat_dump.py, I was able to eat the entirety of #eulora's logs.minigame.biz archive into my test database. Said archive was created via the znc2tmsr code, which is contained within this vpatch along with the fixes to eat_dump.py. Note: the full four year archive took about 40 minutes to eat and resulted in 176887 total loglines.

Special notes on uniturd handling:

As a working example, please take note of index 8412771.

In the 'raw' logs, the 'payload' is as follows:

"Or in my case, re-write a cooking book. So ladies and gentlorans, I give you Foxy’s Euloran Cookbook V1.1,"

Now, in the database, the 'non-standard apostrophe'2 is stored as '\u0092':

nsalog=# select payload from loglines where idx = '841277';
payload
-------------------------------------------------------------------------------------------------------------------
"Or in my case, re-write a cooking book. So ladies and gentlorans, I give you Foxy\u0092s Euloran Cookbook V1.1,"

Now, here's where it got funky for me in testing with reader.py. When reading the output in the 'links' browser, the apostrophe displayed as I would expect:

"Or in my case, re-write a cooking book. So ladies and gentlorans, I give you Foxy’s Euloran Cookbook V1.1,"

Buuut, when I tested in 'lynx' and 'chromium', the apostrophe was omitted entirely (with just a space in its stead):

"Or in my case, re-write a cooking book. So ladies and gentlorans, I give you Foxy s Euloran Cookbook V1.1,"

Same results for other lines that dealt with the 92 hex. So, at this point I'm not entirely sure if this is worth dedicating more time to fix, or if this is something folks can live with. Seems like some browsers will read it, and some will not. I'm open to further futzing if it is deemed a necessity, but if not then I submit this patch for signing.

Various misc threads related to the testing and uniturds (links in reverse chronological order):

flask cache deprecation eggogs; workaround

various testing notes from author; prompt from alf to vpatch

lots of uniturd wrestling/info throughout day; phf drops in with his [algo]

moar uniturd wrestling - 2019-08-15

moar uniturd wrestling - 2019-08-14

moar uniturd wrestling - 2019-08-13

  1. 5 lines down in http://logs.minigame.biz/eulora_log_2015_to_2019_debug.txt []
  2. i.e. not 0x27 []

irclog2html genesis

August 16th, 2019

vpatch: http://lobbesblog.com/src/irclog2html/irclog2html_genesis.vpatch

sighttp://lobbesblog.com/src/irclog2html/irclog2html_genesis.vpatch.lobbes.sig

From :README.txt":

Genesis by lobbes of the heathen "irclog2html-2.14.0" that is currently used to produce

the colorful logs found on http://logs.minigame.biz/


USAGE: Takes .log files from, e.g., ZNC and converts to html. You can then set a cron job

to run the coad and transfer the files to your web-facing directory.


WARNING: NOT recommended for use as-is due to the need for "pip" to install (see:

http://logs.nosuchlabs.com/log/trilema/2019-07-28#1925233). It does come with

a makefile that allows a "test" installation (still via pip) in virtualenv, but I was not able

to get this to work at the time of this genesis due to various "deprecation" barf (this version

was produced in 2014).


The main cause for this genesis is the want to freeze the important bits in amber for future dissection.

I aim to eventually create a vpatch that eliminates the vast majority of this coad and replaces it with

some simple scripts to achieve the same html colorization without the pip-isms. Others, of course, are welcome

to bring their own scalpels and flamethrowers to it.


Important bits:

coad/src/irclog2html/irclog2html.py

coad/src/irclog2html/logs2html.py


From "MANIFEST.TXT":

590462 irclog2html_genesis "Genesis."


Additional notes: The search that is currently in-use over at http://logs.minigame.biz/search/?q= is NOT included in this genesis as that was something I coded after-the-fact. Ditto for the bot long-line echoing done by lobbesbot. I intend to vpatch these at a later date.

znc2tmsr vpatch

August 12th, 2019
Please do not use this vpatch as it is a dead leaf.
Instead, see: http://blog.lobbesblog.com/2019/08/znc2tmsr_etckvvpatch/

Following Diana Coman's slick and effective bash one-liner for converting irssi logs to a format edible by Stan's logotron, I submit for consideration my not-so-slick, yet still effective1 py-turd as a vpatch. MANIFEST.TXT included.

vpatch: http://lobbesblog.com/src/logotron/znc2tmsr.vpatch

sig: http://lobbesblog.com/src/logotron/znc2tmsr.vpatch.lobbes.sig

  1. the jury is still out on if this is indeed what is causing the utf-8 barfs chronicled in the logs. I honestly doubt it, BUT if it is I can always patch again later. Plus, Stan dun have to sign if he dun wanna! That's the magic of V, ain't it? []

Fog of War

August 6th, 2019

War is the realm of uncertainty; three quarters of the factors on which action in war is based are wrapped in a fog of greater or lesser uncertainty.

It is this uncertainty that prompted me to issue my statement on Trilema this morning (as promised, the signed version proving it was from yours truly), and it was Mircea Popescu's prodding to write about myself that prompted me to create this post. So here it is:

Simply speaking, I do not think I am currently capable of annulling the noise around me to direct myself towards +ev republican work. I'm not sure where my hands should go. Unless I'm misunderstanding the proper role of a lord, then I think I'd be best served 'going back to school' and filling the role of a knight of the republic.

Yes, it is true that I want to be able to make these decisions for myself, eventually. I want to be a lord. But wants are different items from facts. I have made my living the last decade as a data analyst, and right now the data is telling me something. Take, for example, the last 12 months of various "hopper updates"; items that I promised to deliver on. Let's do some data analysis: I enumerated roughly 16 distinct items from that set, and out of those 16 I delivered on only 4 of those items. That's only a 25% success rate, or a 75% failure rate on delivery.1

Maybe I'm making too big of a deal of this failure, or maybe not. Maybe I just haven't been communicating enough. After all, the environment is constantly changing by its very nature, and sometimes it is necessary to change focus to adapt to it, sure. However, all that aside I still have the very real problem of lack of technical knowledge and general experience.

I want to make clear something: When I first came through these doors back in 2014, I had no idea what was going on. Even back then when the snr ratio was much lower than today, I only understood about about 10% of what was being said at any given time. The main reason I even followed pankakke's original breadcrumb to the then fledgling republic, and the only reason I kept reading Trilema back then was because I kept witnessing a strange effect happening over and over again: for some reason, all of the things these people were saying kept being proven true. But it went beyond Bitcoin! Indeed, as the years went by, as I toiled through my various meatworld work and interfaced with the various anti-models, I kept seeing the ever-emerging TMSR ideology being proven correct in my everyday life. The more I learned, the more things made sense.

Fast-forward to today and I somehow understand much more of the logs, of the blogs, and what is being discussed. Not to jerk myself off too much here, but I had never touched an OS besides Macintosh/Windows before TMSR. I had never written a lick of python code, nor of php code, nor of lisp, nor of etc. My programming experience was limited to making Macromedia Flash games using "Action Scripting" when I was 16, followed by 5 or so years of nothing until I got my first job out of college.2 Even in my professional life, the most complex 'technology' I have ever really touched is SQL and loading data into the backend of SAP ERP.3 Even worse, these days I don't even get to touch THAT; being confined to toil on various Excel-like toys. My brain would have surely atrophied by now if it weren't for this place.

Nevertheless, it is clear to me that I still have much more to learn. For example, while I've attempted to read the trb source code, I just don't have the c knowledge to grok any of it, really. Hell, after a year's time my own trb node still hasn't fully synced, even with the aggression patch. Obviously this is a very real problem. And while I learned enough about clisp to run the logbot and to build a python command router off of it, I still would never say I know any lisp or clisp at all.4 The list of things I don't know goes on and on, but I guess at least I can say I know enough to know what I don't know, you know?

So, to bring this full circle, I think I need to go back to focusing on learning. I'm going to spend some time in #asciilifeform and #ossasepia, helping the respective lords of those castles out with whatever they need help with. Hopefully I can learn a thing or two in the process. I sure as hell am not going to stop participating however. My goal is to one day get to the point where I have enough knowledge so that I can truly grab my environment by the throat and bend it to my will. But I am not there yet. I have climbed out of the primordial soup, I have learned to crawl and daresay I have learned to walk, but I am not running yet, nor have I come close to taking flight.

And all of the above is just in regards to the public sphere. As far as my private sphere goes, well, I have a pretty clear path there. In the short term I simply have to continue following-through on what I laid out in my most recent forward looking statement. In the long term, well, lets just say that one Trilema post I read in 2018 started out like a majority of them did: first my 'cognitive kill switch'5 kicked in, but then the switch shorted out. The words continued to burn in the back of my head. Slowly burning and burning for months until suddenly it was impossible to ignore. Nevertheless, just like the above, I know what I don't know, and it will be some time before I have the technical knowledge and experience in that realm as well. All in time, I guess, save for that proverbial truck that may one day take me away.

Until then, onward!

  1. Granted, this is a quick n' dirty enumeration, and some of the items I did deliver on (like ditching the dysfunction in my private sphere, or getting a proper Cuntoo workstation working for me) had arguably more weight to them and a greater positive effect on my life than other items in that set. But I'd like to also point out that making a auxiliary logotron was something I promised back in November of 2018. Which... I did not deliver on, and now look where we are! []
  2. Fun fact, I started out college as a CS major, only to switch immediately to Business Management. Partly because I genuinely enjoy studying business, but mostly because I believed it to be the 'path of least resistance' to a degree that would net me a high salary. Sadly, I was right. I 'exam took' the shit out of that thing... and was high for most of it. Oh well, wasted youth []
  3. LSMW and BDC macros ftw! []
  4. and fuck me if you ask me to delineate the difference between clisp and lisp! Even though I've read the threads in the logs, I just can't fully grok it yet []
  5. ty tlp for that coinage, and fuck you Great Inca for putting it there in the first place []

Btcbase.org snarf: first run

August 2nd, 2019

In response to: http://trilema.com/2019/trilema-goes-dark/#comment-130477

SIGNED plaintext version of below text is available at: http://blog.lobbesblog.com/static/btcbase_snarf_attempt1.asc


On 2 Aug. 2200 hrs (NYC time), I generated a snapshot of Phf's btcbase.org/log/, using the method outlined in http://nosuchlabs.com/pub/snapshot_signed.txt :

'wget --recursive --no-clobber --page-requisites --html-extension --convert-links --domains btcbase.org --reject log-search  btcbase.org/log/'

Resulting item :

btcbase_scrape.tar.gz

sha512=7a6e62f2f1c456ea2d17596434b8acf86fbd4f76e3cd78795075a3a296aa7d8e5aebf8e63b00145928e6ad61be4b3742bca3068794c00ffdd452c993a3d9fb3e

This has been made available at:

http://blog.lobbesblog.com/static/btcbase_scrape.tar.gz

NOTE: The sha512 between this and asciilifeform's original snarf (http://nosuchlabs.com/pub/btcbase_snap.tar.gz) DIFFERS

Standard diff between the two has been run and made available at:

http://blog.lobbesblog.com/static/btcbase_snap_diff.txt

Please see following thread for more details: http://logs.bvulpes.com/asciilifeform?d=2019-8-3#476022

I will be doing a second run with proper wget error logging as well as rate-limiting to rule out simple time-out issues. More to come.

Time estimation for logotron based off of logbot: August 18th

July 28th, 2019

This weekend I came to the conclusion that the current #eulora logotron must die, and that I will replace this with a more proper1 logotron based off of logbot. The problem here is that diana_coman has been busy training new hands, and the benefits of this to the republic could be further amplified if only she had a working logotron.

So, as promised, here is what I was able to spec, and my estimation of how long it will take me to implement said spec.

Current estimated time-to-delivery: August 18th, 2019

Specifications:

  • Based off of the log table in the Postgres database used by the logbot. Will need to make a separate Postgres schema that will be web facing, with php-olade on top that grabs the data needed to display teh loglines.
  • Will need to segment the log by day, with the ability to go to the previous/next days
  • Must be able to search log
    • Allow the "From:nick" syntax in the search
    • Allow to search "with quotes" or without quotes2

NOT included in this spec:

  • Bot log quoting. This will have to wait until later as I want to think this one through a bit more.


  1. Depending who you ask, I guess. I'm not a doctor in this hospital, however, so I honestly couldn't tell you for sure. []
  2. There's probably a term for this but it is almost midnight here and I don't care THAT much; you get the point I'm sure []

Forward-looking report: Weeks 28-31 of 2019

July 7th, 2019

Since the previous report, I managed to hit my goal of loading 100 more lines of the mod lisp coad into my head. For my practical work plan, I will continue to hit this goal of ~100 lines per week for the next 4 weeks.

However, it has become clear to me that I need to remedy some things in my private sphere as a priority, before I can really work/grow at the level I want. It has become clear to me that in order for me to start taking my own life back, I need to start at my home base of operations. Simply scheduling my TMSR work is not enough to fix my real issue.

I'll save the (boring) details for a later, more reflective post after-the-fact, but for now let's just say I need to make some small (and long-overdue) logistical changes in my life that will require a lot my attention in the short-term , with the aim that it'll both greatly reduce my expenses, and greatly increase my free time in the long-term.

Forward-looking report: Week 27 of 2019

June 30th, 2019

It is said that in every functional system, the buck must stop with a single authority, lest everything melt into a bureaucratic soup. Well, in the logs of June 22nd the authority of the world's only sovereign was not happy, and knocked most of the L1 over the head for lack of clear, forward-looking communication.

Indeed, I am guilty of recently engaging in the described orc-activity, so I will try to address this via weekly forward-looking reports on what I intend to do that week.


We are about to enter week 27 of 2019. The current item in my conveyor is chewing through the mod_lisp coad. I've mentioned it in the logs (and on my blog), but I'll say it again here: I am 100% new to C, so I am putting my face to the cheese grater (to borrow a hanbot-ism) as I eat each line. It is slow going but I'm grokking enough to be able to answer to the coad once I sign it (which is my goal here).

Still, it'll probably go faster if I pull my head out of my ass and actually structure my learning/reading. As such, I've 'loaded in my head' up to line 185 so far1, so I'll aim to grok up to ~line 285 by next week's report. I've got a lot of $saltmine work coming up in the next two weeks, but somehow I need to find a way to get real work done. Let's see if this small chunks thing works.

  1. which, admittedly, is pretty sad for a month's time... []

Hopper Update: May 2019

May 16th, 2019

Since January's hopper update, I managed to get Cuntoo working for me and released the vpatch for auctionbot. Next up:

cl-wwwism kit - There's definitely a need for some basic www interfacing tools to be genesis-ed, and to be frank I'm not sure atm what the 'final kit' might look like exactly. However, this may not be a problem right now because the v-approach to things seems to be a piecemeal approach; building yggdrasil as it were. In any case, spyked has outlined for us some basic elements for such a kit:

  • http server
  • curl
  • html gen
  • So I decided to attack the http server piece and am currently chewing through the mod_lisp coad,1 and once it fits in my head I will poop out a genesis.

    logotronics - This item depends on some of the www interfacing items mentioned above, namely: the html generator and http server. However, it'll also require some Postgres interfacing to retrieve loglines from the log table that logbot uses. Once everything exists I will spin up a few auxiliary logotrons for channels like #pizarro and #trilema (or any others on request), as well as look into Mr. P's request re:

    mp_en_viaje: lobbes, can you do that ? make your logger 1) log ~every irc channel over 50 users~, 2) produce this data upon request by a lord only, and for pay ?

    other items - Once the above items start taking more shape, I aim to circle back to the remaining items in http://blog.lobbesblog.com/2018/11/conveyor-outlook-now-to-feb-2019/

    1. Which is taking me time since I know not C nor really Common Lisp yet, so it is currently a 'line-by-line with the respective reference manuals in-hand' exercise. []

    Auctionbot: vpatch and manifest

    April 26th, 2019

    patch: logbot_command_router_python_auction.vpatch

    signers: lobbes

    This vpatch extends logbot command router to add auction functionality and builds upon the Keccak regrind of the botworks V tree. As I understand it, here is what that tree looks like today with the addition of this vpatch:

    botworks-v-tree-2019-04-27.svg

    Press logbot_command_router_python_auction via your preferred V:

    mkdir botworks; cd botworks; mkdir .wot

    #You'll need the keys for both lobbes and spyked in .wot

    vk.pl init http://lobbesblog.com/src/botworks/

    vk.pl p v logbot_command_router_python_auction logbot_command_router_python_auction.vpatch

    From the INSTALL:

    ========

    INSTALL:

    After pressing and assuming you have already configured the command router (see the parent README), all you need to do is run the auctionbot.sql against the postgres database used by the command router. This will create all of the auction tables (they all will be prepended with "auct_"). From there, just start up the command router (i.e. ./main.py) and all of the auction commands should function (creating auctions, bidding, etc.)

    example:

    su postgres -c 'psql -f auctionbot.sql postgres'

    ========

    IMPORTANT NOTE:

    This bot does not have a SCHEDULER baked in (for e.g. listing auctions out in a channel every X hours). This was intentional, as I wanted to leave that portion up to the operator.

    Nevertheless, it is CRUCIAL you schedule a recurring call of at least the 'resolve' command, as this will ensure all of the auctions are properly resolved and closed as they expire. I recommend calling this once every minute.

    How I have personally handled this is I have a few cron jobs that run INSERT statements against the postgres database every X interval. Examples below.

    -Example SQL to essentially call the 'list' command 'from' #lobbestest:

    INSERT INTO log(target, message, source) VALUES ('#lobbestest','?Xlist','auct_cron');

    What this does is insert a record into the log table used by the parent logbot. The command router sees this as the command 'list' coming from '#lobbestest' issued by the nick 'auct_cron', and responds by listing in that channel.

    Some may see this as 'tainting' the log table with 'false' loglines, but if you are thoughtful about it you can make it so those records can be easily filtered out of, say, a logotron's display (a 'reporting' database ought to be separate from a 'production' database anyways, imho).

    -Then I simply add these 'one-liner' sql scripts to run on a cron job. Here's one of my crontabs, for example:

    * * * * * su postgres -c 'psql -f /home/lobbes/auctionbot/auct_schedule_cmd_resolve.sql postgres'

    0 * * * * su postgres -c 'psql -f /home/lobbes/auctionbot/auct_schedule_cmd_list_lobbestest.sql postgres'

    0 */5 * * * su postgres -c 'psql -f /home/lobbes/auctionbot/auct_schedule_cmd_list_eulora.sql postgres'

    0 */5 * * * su postgres -c 'psql -f /home/lobbes/auctionbot/auct_schedule_cmd_list_pizarro.sql postgres'

    0 */5 * * * su postgres -c 'psql -f /home/lobbes/auctionbot/auct_schedule_cmd_list_trilema-lobbes.sql postgres'

    From the manifest:

    424545 ircbot-genesis trinque ircbot genesis, http://trinque.org/2016/08/10/ircbot-genesis/
    424689 logbot-genesis trinque logbot genesis, http://trinque.org/2016/08/11/logbot-genesis/
    447104 ircbot-logbot-multiple-channels-corrected ben_bulpes multiple channel support for ircbot and logbot, http://cascadianhacker.com/correction-multiple-channel-patches-for-irclogbot
    539566 logbot_command_router_python_genesis lobbes logbot command router for python, http://blog.lobbesblog.com/2018/09/logbot_command_router_python-genesis/
    573368 logbot_command_router_python_auction lobbes auction commands for logbot command router, http://blog.lobbesblog.com/2019/04/auctionbot-vpatch-and-manifest/