Seeking Forum Input: Auctionbot's Currency - btc vs ecu

September 9th, 2018

UPDATE: See thread (http://btcbase.org/log/2018-09-09#1848940). Question answered. Input closed. Bot will deal 100% in ecu.

=============================

Now that I've released the command router for logbot, I've turned my attention to really trying to think through the (re)design of auctionbot. I'm currently at the point where I need to make a couple of design decisions and would like to field some input from the forum.

Question 1: Should the auctionbot deal in btc or ecu? (e.g. "!Xbid [ecu amount]" vs "!Xbid [btc amount]")

Question 2: Should the auctionbot accept bid/order amounts in either currency?1

I'm current thinking is that:

Answer to 1) I will have the thing deal in btc as opposed to ecu

Reasoning being that I need to pick one currency, and trb is the currency of the republic

Answer to 2) I will not offer any automatic btc-to-ecu conversion features.

Reasoning being I don't want to be in business of curating conversion rates for this. I'm thinking of cases like: I'm vacationing on the moon for a month. During that time ecu to btc rate changes but the bot isn't updated to reflect this change. People don't notice and chaos ensues2

Does this sound like solid reasoning? And would eulora players hate to enter their auctions in btc vs ecu?

  1. i.e. the bot converts input/output where needed to/from the currency chosen in  question 1 []
  2. e.g. someone auctions off something for their intended amount, but bot incorrectly converts it into some other amount. I'm then stuck having to unwind data after-the-fact []

logbot_command_router_python: Genesis

September 2nd, 2018

Special thanks to ben_vulpes for helping me understand the fundamental bits of the design that this specific implementation was based on.

What is this?

This is a package of .py scripts designed to 1) hook into a PostgreSQL db that is being used by a running instance of logbot or logbot-multiple-channels-corrected in order to 2) act as a configurable and extendable 'bot command router'

Where is this?

From my repository:

genesis vpatch (lobbes sig) (spyked sig)

Alternatively, see spyked's Keccak regrind for the patches and this post for my signature

This:

From INSTALL:

Install the following via your preferred method: 

logbot-genesis or logbot-multiple-channels-corrected (http://btcbase.org/patches?patchset=bot&search=)
psycopg2 (a PostgreSQL adapter for Python: http://initd.org/psycopg/)

Press logbot_command_router_python via your preferred V:

mkdir -p ~/src/logbot_command_router_python
cd ~/src/logbot_command_router_python

mkdir .wot
cd .wot && wget http://www.lobbesblog.com/lobbes.asc && cd ..

v.pl init http://www.lobbesblog.com/src/logbot_command_router_python
v.pl press logbot_command_router_python logbot_command_router_python_genesis.vpatch

From README:

HOW TO USE:

Bits designed to be customized to your needs:
knobs/config.py << Edit you postgres db connection info, bot command prefix, etc.
knobs/router.py << The central command router. Write or pull in your custom commands here.
commands/* << A directory to house command scripts that you can import into router.py

To run:
Configure your bits mentioned above
Start your local postgres server and your instance of logbot or logbot-multiple-channels-corrected
./main.py

CAVEATS:

Designed to communicate with a LOCAL Postgres server only, not remote. (Feel free to extend this to add that capability, however)

Auctionbot ETA and Status Report

July 28th, 2018

This post will serve to provide communication on the overall status of Auctionbot. I will update periodically as I see fit.

Current ETA1 : October 31st, 2018

Steps to Fruition:

  1. [Complete] Spin up instance of logbot-multiple-channels-corrected and confirm functionality
  2. [Complete] Successful test run of an external .py script communicating on IRC via logbot and the listen/notify mechanisms of postgres. Baked the polished tests into a logbot-command-router-genesis
  3. [Complete] Re-design and re-implement auctionbot as an extension of logbot-command-router-genesis, leveraging the legacy2 auctionbot coad where possible. It will include the reverse auction functionality specified in the logs.
  4. [Complete]Test.
  5. [Complete]Release into production

-

  1. I will announce in #trilema if this date is pushed out []
  2. i.e. the auction 'plugin' I wrote for lobbesbot []

logbot-multiple-channels-corrected on Gentoo: Tips n' Tricks for the Uninitiated

July 20th, 2018

Recently I stood-up an instance of logbot-multiple-channels-corrected on my Pizarro Rockchip.1 As such, I accumulated a large pile of notes that I cobbled into this list of "tips n' tricks". While I'm no talking paperclip, I hope this will be of use to folk in the audience who may be new to any or all of: Gentoo, v, postgresql, lisp, whathaveyou.

We will be covering the following:

  • sbcl & Quicklisp
  • PostgreSQL (with uuid USE flag)
  • ircbot-genesis & ircbot-multiple-channels-corrected
  • logbot-genesis & logbot-multiple-channels-corrected

sbcl & Quicklisp

Both are needed and both are pretty straightforward installation-wise.2 As such, I'm just going to provide a pointer to the Quicklisp installation page and leave it at that.

PostgreSQL (with uuid USE flag)

> As the parenthetical in the above heading suggests, you need to set your USE flag for "uuid" before you emerge Postgres as the logbot.sql you will run later requires the uuid-ossp module.

> Also be wary of your PYTHON_SINGLE_TARGET flag when emerging3 as mine defaulted to 3_5. You can set your PYTHON_SINGLE_TARGET via your package.use:

dev-db/postgresql PYTHON_SINGLE_TARGET: -* python2_7

> While I think this depends on -how- you install, my installation4 created a database cluster with three databases: template0, template1, and postgres. From what I've learned, the 'postgres' database (owned by the 'postgres' superuser created by the installation) is meant to be used as a general purpose database. The 'template1'  database is used as a template when creating new databases, and 'template0' is primarily used to restore template1 in the event you take a dump on it.

> Once installed, your logs and config files will be located somewhere like here:

configs:

/etc/postgresql-9.5/postgresql.conf

logs:

/var/lib/postgresql/9.5/data/postmaster.log

> For postgres-9.5 at least, this is how to start/stop the server:

As root:

/etc/init.d/postgresql-9.5 start

/etc/init.d/postgresql-9.5 stop

> Connecting to the default 'postgres' database (as 'postgres' user):

pizrk003 portage # su postgres -c 'psql'

psql (9.5.12)

Type "help" for help.

postgres=#

> Link to the docs: https://www.postgresql.org/docs/

ircbot-genesis & ircbot-multiple-channels-corrected

> For ircbot-genesis, follow the INSTALL steps on btcbase.org/patches as it will take you most of the way. I also needed to load Quicklisp before 'cl-irc' in the SBCL REPL:

(load "~/quicklisp/setup.lisp")

(ql:quickload :cl-irc)

> Also be on the lookout for a rogue "robots.txt" file that ends up in the .seals directory on the 'init' step of the INSTALL. If you get the following barf on press5 then just remove it from .seals:

lobbes@pizrk003 ~/src/ircbot $ ~/v/v.pl press ircbot-genesis ircbot-genesis.vpatch

----------------------------------------------------------------------------------

WARNING: robots.txt is an INVALID seal for ircbot-genesis.vpatch!

Check that this user is in your WoT.

Otherwise remove the invalid seal from your SEALS directory.

----------------------------------------------------------------------------------

Died at /home/lobbes/v/v.pl line 594.

lobbes@pizrk003 ~/src/ircbot $ ls /home/lobbes/src/ircbot/.seals

ircbot-genesis.vpatch.trinque.sig  robots.txt

lobbes@pizrk003 ~/src/ircbot $ rm /home/lobbes/src/ircbot/.seals/robots.txt

> For ircbot-multiple-channels-corrected, you can do something like:

cd ~/src/ircbot/.wot &&

wget -O ben_vulpes.asc http://wot.deedbot.org/4F7907942CA8B89B01E25A762AFA1A9FD2D031DA.asc

cd ../patches &&

wget -O ircbot-multiple-channels-corrected.vpatch http://btcbase.org/patches/ircbot-multiple-channels-corrected/file

cd ../.seals &&

wget -O ircbot-multiple-channels-corrected.vpatch.ben_vulpes.sig http://btcbase.org/patches/ircbot-multiple-channels-corrected/seal/ben_vulpes

cd ../ &&

v.pl press ircbot-multiple-channels-corrected ircbot-multiple-channels-corrected.vpatch

> Remember to update your symlink for quicklisp:

ln -sfn ~/src/ircbot/ircbot-multiple-channels-corrected ~/quicklisp/local-projects/ircbot

logbot-genesis & logbot-multiple-channels-corrected

The steps for logbot-genesis and logbot-multiple-channels-corrected are similar to ircbot, with the obvious differences relating to interfacing with postgresql.

> You'll want to edit your USAGE to look something like this6. (Note: If you are connecting to the postgres database, you do not specify a pw. In which case, leave that blank in your USAGE file):

(asdf:load-system :logbot)

(defvar *bot*)

(setf *bot*

(logbot:make-logbot

"chat.freenode.net" 6667 "nick" "password"

'("#channel1" "#channel2")

'("db-name" "db-user" "db-password" "db-host")))

; connect in separate thread, returning thread

(logbot:ircbot-connect-thread *bot*)

; or connect using the current thread

; (logbot:ircbot-connect *bot*)

> Similarly to ircbot, don't forget to update your quicklisp symlinks to point to logbot-multiple-channels-corrected:

ln -sfn ~/src/logbot/logbot-multiple-channels-corrected ~/quicklisp/local-projects/logbot

> To start the bot, you'll want to load the following from the SBCL REPL:

(load "/[path]/[to]/quicklisp/setup.lisp")

(ql:quickload :cl-irc)

(ql:quickload :cl-postgres)

(ql:quickload :postmodern)

(load "[Your configured USAGE file referenced above]")

> Checking your LOG table to see logged IRC lines:

pizrk003 lisp # su postgres -c 'psql'

psql (9.5.12)

Type "help" for help.

postgres=# SELECT * FROM LOG;

id                  |    target    |  message   |      host       | source |  user   |        received_at

--------------------------------------+--------------+------------+-----------------+--------+---------+----------------------------

8061cf9a-1e0b-4a3c-a0d9-837afa8cae55 | #lobbestest  | test       | 192.121.170.137 | lobbes | ~lobbes | 2018-07-16 03:05:06.698238

ab8fc726-8c46-4495-b571-64789de02826 | #lobbestest  | test five  | 192.121.170.137 | lobbes | ~lobbes | 2018-07-16 03:05:11.140429

21f03cb6-3322-4332-af85-133d742113a0 | #lobbestest  | test 1234  | 192.121.170.137 | lobbes | ~lobbes | 2018-07-16 03:30:56.743491

b6d7fa3f-1e84-4fde-9e4d-b3dd7e2b8d32 | #lobbestest2 | test'      | 192.121.170.137 | lobbes | ~lobbes | 2018-07-16 05:49:53.669309

cd48a6c0-dc8e-449e-bc68-5a3aabc16c48 | #lobbestest  | test       | 192.121.170.137 | lobbes | ~lobbes | 2018-07-16 05:49:57.800388

2e7c2bb6-e9c4-41fb-a629-1a0f5e1682ba | #lobbestest2 | test again | 192.121.170.137 | lobbes | ~lobbes | 2018-07-16 05:56:07.579301

96b6d216-44e5-4b4d-99b8-c136fb538b27 | #lobbestest  | test niaga | 192.121.170.137 | lobbes | ~lobbes | 2018-07-16 05:56:30.749328

  1.  "Auctionbot 2.0" will sit on top of this. Look soon for a guide on feeding commands to/from logbot-multiple-channels-corrected via postgres triggers []
  2. actually, I think my P-rockchip came with sbcl already emerged []
  3. I've quickly learned that the "-a" option is your friend when it comes to "emerge" []
  4. which was via portage using: a package mask on ">dev-db/postgresql-9.5.12"; "postgresql" and "uuid" USE flags in "make.conf"; then "emerge -av dev-db/postgresql" []
  5. this barf, of course, may look different depending on your V []
  6. pardon the lack of indents; blog formatting woes []

Sales Report: June 2018

July 7th, 2018

Since being reminded that proper reporting is a primary element of the action/report/discuss/adjust cycle, I figure I will start releasing monthly reports of my actions towards building sales for Pizarro ISP. And with that short intro, here is the June report which documents the actions I have taken since being given the green light to begin sales on June 3rd. :

Sales brought in this month1

0

Summary of Activity

Since I had to start somewhere, I figured I'd survey the layout of two corners of the vast refuse pile: the "webhosting forums" and the "bitcoin forums".

As it turns out, a good chunk of the webhosting forums out there have certain thresholds2 that need to be met in order to post "advertisements" on their designated advertisement boards. I targeted what usgoogle claims is the most relevant result for the query of "web hosting forums"3 and set off to reach that forum's threshold of "10 non-fluff" posts. While seemingly simple, this proved to be an expensive4 exercise; eating through minutes and brain cells as I stepped on "gotcha" landmines such as "too fluffy a reply, post deleted" and "posts in X board don't count toward post count". Even after hitting the threshold and posting my ad, I stepped on another landmine of "post deleted due to incomplete public whois information"

As you might expect, I was met with no such "threshold to post ads" on the various "bitcoin forums" and was able to post ads immediately upon account creation5. I also got two kinda-leads from the post on bitcointalk, which I pursued to no results6

If any are interested, I am keeping a running list of the ads I have here: http://blog.lobbesblog.com/list-of-pizarro-ads/

Takeaway

In my eyes, I utterly have failed and flailed thus far. Not much to show here, and it shows. Still, I got some good direction from the forum, both directly and indirectly7. My plan of attack for this next month:

  • Utilize tracking on my ads so as to get proper measurements. I have already retrofitted my existing ads with links pointing to variations of the url of my custom landing page
  • Look for additional avenues outside of the "web hosting forums".
  • Automation: Explore ways to construct a mass-mailer of sorts; figure out the proper design of a trawler8

Onward.


  1. and year-to-date []
  2. mostly "N post-count required" []
  3. at the time of this writing this is "www.webhostingtalk.com" []
  4. considering this is effort expended to post a single thread that will be seen by maybe.. ~20 people before sinking to the 2nd page of roughly 100 entries in a digital phonebook that I suspect nobody consults []
  5. granted, for tardstalk I didn't have to start 'fresh' as I had an old account that I simply dusted off []
  6. perhaps opening with a sales pitch on that second lead was a bad idea. []
  7. as oft the case []
  8. and where to trawl []

Arming your Arm64 RockChip Gentoo against the hordes of Mindless Bots

July 1st, 2018

About a week ago I was notified in #trilema that my blog was unreachable. After being convinced to resist the urge to engage in the usual shamanism, I set off to first define the problem properly.

So, in an attempt to diagnose I cut the patient back open and stuck my hands in the chest cavity to feel around. Let's look closely at the apache error logs1 :

[Tue Jun 26 05:42:31.703938 2018] [mpm_prefork:error] [pid 12260] AH00161: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting

Looks like apache is hitting the maximum number of workers. Let's see what netstat is saying:

pizrk003 htdocs # netstat -an | egrep ':80|:443' | grep ESTABLISHED | awk '{print $5}' | grep -o -E "([0-9]{1,3}[\.]){3}[0-9]{1,3}" | sort -n | uniq -c | sort -nr

187 173.254.216.66

Here we see one IP address is slamming my blog with 187 unique requests (all ESTABLISHED state). A quick look at the apache config file confirms that this many requests indeed exceeds the max limit:

pizrk003 htdocs # cat /etc/apache2/modules.d/00_mpm.conf | grep 'MaxRequestWorkers'

# MaxRequestWorkers: Maximum number of child processes to serve requests

MaxRequestWorkers       150

MaxRequestWorkers       150

MaxRequestWorkers       150

MaxRequestWorkers       150

MaxRequestWorkers       150

Okay, so now we have the problem formally defined as "Apache cannot serve requests due to MaxRequestWorkers limit being exceeded".

Now to fix. Let's first try the obvious treatment of simply increasing the MaxRequestWorkers in the apache config to something like 256. Restart apache, and then netstat again:

pizrk003 htdocs # netstat -an | egrep ':80|:443' | grep ESTABLISHED | awk '{print $5}' | grep -o -E "([0-9]{1,3}[\.]){3}[0-9]{1,3}" | sort -n | uniq -c | sort -nr

256 94.230.208.147

Sadly, no dice. Now the spamola has simply increased to hit the new max (and from a new ip, no less). Looks like we will need to do something to limit spam requests from getting through. Unfortunately for us, a firewall solution is not readily available as iptables does not function "out-of-the-box" on this thing. Nevertheless, there might be a few things we can try at the apache layer for the time-being.

Mod_evasive

The module "mod_evasive" works by giving apache the functionality to automatically deny any ip address that requests X resource more than N times per I interval. Here's how I got this up and running:

First, add "www-apache/mod_evasive-1.10.1 ~arm64" to your package.accept_keywords file (or directory, if you prefer it that way)

echo '=www-apache/mod_evasive-1.10.1 ~arm64' >> /etc/portage/package.accept_keywords

For my fellow Gentoo (and arm architecture) n00bs out there, let me take a short interlude to explain what I've come to understand about how keywords work in Gentoo, and why we need to do the above in our case2.

From https://wiki.gentoo.org/wiki/KEYWORDS:

In an ebuild the KEYWORDS variable informs in which architectures the ebuild is stable or still in testing phase.

And from https://wiki.gentoo.org/wiki/ACCEPT_KEYWORDS:

The ACCEPT_KEYWORDS variable informs the package manager which ebuilds' KEYWORDS values it is allowed to accept.

...

Stable and unstable keywords

The default value of most profiles' ACCEPT_KEYWORDS variable is the architecture itself, like amd64 or arm.

In these cases, the package manager will only accept ebuilds whose KEYWORDS variable contains this architecture.

If the user wants to be able to install and work with ebuilds that are not considered production-ready yet, they can add the same architecture but with the ~ prefix to it, like so:

FILE /etc/portage/make.conf

ACCEPT_KEYWORDS="~amd64"

This, I think, is important for us to understand because mod_evasive is definitely not listed as stable for the arm64 architecture. That being said, this Gentoo's make.conf had the value of "**" for the ACCEPT_KEYWORDS, which, I think may already tell Portage to pull in untested/unstable packages and may also override whatever you set in package.accept_keywords. In other words, this may be an exercise in redundancy, but at least we all learned something!3.

Interlude over; let's keep moving. After getting your keywords situated, let's slap the "mod_evasive" USE flag in make.conf and emerge:

pizrk003 lobbes # emerge -av www-apache/mod_evasive

[...SNIP...]

These are the packages that would be merged, in order:

Calculating dependencies... done!

[ebuild  N     ] www-apache/mod_evasive-1.10.1-r1::gentoo  20 KiB

Total: 1 package (1 new), Size of downloads: 20 KiB

Would you like to merge these packages? [Yes/No] Yes

>>> Verifying ebuild manifests

>>> Emerging (1 of 1) www-apache/mod_evasive-1.10.1-r1::gentoo

>>> Installing (1 of 1) www-apache/mod_evasive-1.10.1-r1::gentoo

>>> Recording www-apache/mod_evasive in "world" favorites file...

>>> Jobs: 1 of 1 complete                           Load avg: 0.41, 0.12, 0.04

>>> Auto-cleaning packages...

>>> No outdated packages were found on your system.

* GNU info directory index is up-to-date.

[...SNIP...]

Seems good. And looky here, we now have a mod_evasive config file to edit:

pizrk003 lobbes # more /etc/apache2/modules.d/10_mod_evasive.conf

<IfDefine EVASIVE>

LoadModule evasive_module modules/mod_evasive.so

DOSHashTableSize 3097

DOSPageCount 5

DOSSiteCount 100

DOSPageInterval 2

DOSSiteInterval 2

DOSBlockingPeriod 10

# Set here an email to notify the DoS to someone

# (here is better to set the server administrator email)

DOSEmailNotify root

# Uncomment this line if you want to execute a specific command

# after the DoS detection

#DOSSystemCommand    "su - someuser -c '/sbin/... %s ...'"

# Specify the desired mod_evasive log location

DOSLogDir /var/log/apache2/evasive

# WHITELISTING IP ADDRESSES

# IP addresses of trusted clients can be whitelisted to insure they are never

# denied.  The purpose of whitelisting is to protect software, scripts, local

# searchbots, or other automated tools from being denied for requesting large

# amounts of data from the server.

#DOSWhitelist    127.0.0.*

#DOSWhitelist    172.16.1.*

</IfDefine>

# vim: ts=4 filetype=apache

Tweak this how you like. Here's a quick rundown on the interesting knobs:

DOSPageCount and DOSSiteCount: Amount of requests allowed per interval to unique pages and the site as a whole, respectively.

DOSPageInterval and DOSSiteInterval: Sets the interval (in seconds) for unique pages and the site as a whole, respectively.

DOSBlockingPeriod: Amount of time (in seconds) for the ip to be denied.

DOSWhitelist: Pretty self-explanatory, but allow specific IP addresses from being exempt from the above restrictions.

Let's also edit the apache config4 to load the mod_evasive module. Add  "-D EVASIVE" to the "APACHE2_OPTS" line:

APACHE2_OPTS="-D DEFAULT_VHOST -D INFO -D SSL -D SSL_DEFAULT_VHOST -D LANGUAGE -D PHP -D EVASIVE"

Now restart apache and do some stress-testing to confirm everything's working. Tweak your configuration as necessary.

Mod_limitipconn

While mod_evasive has utility, it didn't really address my particular problem, which was a single ip address exceeding the max amount of apache workers. Since the ips spamming my box were sending their requests at reasonable intervals5 mod_evasive wasn't much use unless I wanted to also block legit traffic. Enter mod_limitipconn. As the name suggests, this module gives apache the ability to limit unique ip addresses to a set number of max requests. Let's get it emerged and configured.

Like with mod_evasive, add the required bits to your make.conf, apache config, and package.keywords for mod_limitipconn:

echo '=www-apache/mod_limitipconn-0.24 ~arm64' >> /etc/portage/package.accept_keywords

=======================

cat /etc/portage/make.conf

[...SNIP...]

USE="mysql cgi php apache2 fpm apachetop mod_evasive mod_limitipconn -gtk3 -avahi -gnome -tls-heartbeat -gpm -X -libnotify -consolekit offensive ufw -dbus -bluetooth -systemd -wayland$

[...SNIP...]

=======================

cat /etc/conf.d/apache2

[...SNIP...]

APACHE2_OPTS="-D DEFAULT_VHOST -D INFO -D SSL -D SSL_DEFAULT_VHOST -D LANGUAGE -D PHP -D EVASIVE -D LIMITIPCONN"

[...SNIP...]

Now, I also had to add the following package mask to force Portage to pull in the earlier version of this thing6:

pizrk003 lobbes # more /etc/portage/package.mask/mod_limitipconn

>www-apache/mod_limitipconn-0.24

Now, emerge -av www-apache/mod_limitipconn-0.24. It should go smoothly.

Important: for this module to function, you will also need mod_status enabled with the "ExtendedStatus On". On this box, mod_status was already installed7, so we just need to define it by adding "-D STATUS" to the "APACHE2_OPTS=" line in the apache config:

"APACHE2_OPTS="-D DEFAULT_VHOST -D INFO -D SSL -D SSL_DEFAULT_VHOST -D LANGUAGE -D PHP -D EVASIVE -D LIMITIPCONN -D STATUS"

Now, time to configure mod_limitipconn by editing /etc/apache2/modules.d/27_mod_limitipconn.conf. I found this to be slightly tricky. The following worked for me, so feel free to use it as a general guide8:

<IfDefine LIMITIPCONN>

LoadModule limitipconn_module modules/mod_limitipconn.so

<Location /var/www/localhost/htdocs/>

MaxConnPerIP 10

# exempting images from the connection limit is often a good

# idea if your web page has lots of inline images, since these

# pages often generate a flurry of concurrent image requests

NoIPLimit blog/images/*

</Location>

<Location /mp3>

MaxConnPerIP 1

# In this case, all MIME types other than audio/mpeg and video*

# are exempt from the limit check

OnlyIPLimit audio/mpeg video

</Location>

<IfModule mod_limitipconn.c>

# Set a server-wide limit of 10 simultaneous downloads per IP,

# no matter what.

MaxConnPerIP 10

</IfModule>

</IfDefine>

Let's see if it works. One quick restart of apache and then netstat:

pizrk003 htdocs # netstat -an | egrep ':80|:443' | grep ESTABLISHED | awk '{print $5}' | grep -o -E "([0-9]{1,3}[\.]){3}[0-9]{1,3}" | sort -n | uniq -c | sort -nr

25 197.231.221.211

Mwahah, 25 much better than 256, huh? The 'raw' netstat I issued right before this was showing some 300 connection attempts from this same ip, but with the status of "TIME_WAIT" and "CLOSE_WAIT" instead of "ESTABLISHED". Sure enough, a few seconds later another netstat reveals:

pizrk003 htdocs # netstat -an | egrep ':80|:443' | grep ESTABLISHED | awk '{print $5}' | grep -o -E "([0-9]{1,3}[\.]){3}[0-9]{1,3}" | sort -n | uniq -c | sort -nr

Nothing! The robots have stopped knocking.

Hopefully this will be of use to you if you are trying to carve a webserver out of the Arm64 RockChip Gentoo. Now, if I can just get iptables working...

  1. located in /var/log/apache2/error_log []
  2. Gentoo experts out there, plox to correct me here []
  3. maybe? []
  4. located at /etc/conf.d/apache2 []
  5. one per second, roughly []
  6. as the "0.24-r2" version dun seem to work []
  7. see the config in  /etc/apache2/modules.d/00_mod_status.conf []
  8. if you want more info, please see this handy README I found: http://dominia.org/djao/limitipconn2-README  []

Getting PHP5.6 and MySql to Play Nice On Arm64 RockChip Gentoo

June 20th, 2018

This blog used to live on a heathen VPS until the limitations of using a toy-server were finally felt. As luck would have it, I had just procured a spiffy new Rockchip from Pizarro ISP, so off I set to resurrect lobbesblog on this beautiful Gentoo arm64 box.

I'm absolutely a Gentoo n00b, but I was able to get 'AM' portions of LAMP emerged without too much trouble, and 'P' with help from BingoBoingo's helpful guide. The problem I ran into, however, was getting the 'M' and the 'P' playing nice. Specifically: Portage would not emerge the 'mysql module' for php no matter how hard I typed 'mysql' for the USE flags in make.conf and/or package.use.

The path to my solution was revealed once I paid attention to a specific piece of the problem output:

pizrk003 blog # emerge --ask dev-lang/php:5.6

* IMPORTANT: 4 news items need reading for repository 'gentoo'.

* Use eselect news read to view new items.

* Last emerge --sync was 47d 12h 28m 3s ago.

* IMPORTANT: config file '/etc/portage/package.use' needs updating.

* See the CONFIGURATION FILES and CONFIGURATION FILES UPDATE TOOLS

* sections of the emerge man page to learn how to update config files.

These are the packages that would be merged, in order:

Calculating dependencies... done!

[ebuild     UD ] dev-libs/oniguruma-6.7.1 [6.8.1] USE="-combination-explosion-check%"

[ebuild  NS    ] dev-lang/php-5.6.35-r1 [7.2.4] USE="acl apache2 berkdb bzip2 cgi cli crypt ctype fileinfo filter fpm gdbm hash iconv ipv6 json mysqli nls opcache phar posix readline session simplexml ssl tokenizer unicode xml zlib -bcmath -calendar -cdb -cjk -coverage -curl -debug -embed -enchant -exif (-firebird) -flatfile -ftp -gd -gmp -imap -inifile -intl -iodbc (-kerberos) (-ldap) (-ldap-sasl) -libedit -libmysqlclient -libressl -mhash (-mssql) (-mysql) (-oci8-instant-client) -odbc -pcntl -pdo -postgres (-qdbm) -recode (-selinux) -sharedmem -snmp -soap -sockets -spell -sqlite -sybase-ct -systemd -sysvipc -threads -tidy -truetype (-vpx) -wddx -xmlreader -xmlrpc -xmlwriter -xpm -xslt -zip"

"Hey, so what the hell do those brackets mean?" was the question I asked myself that eventually led me to the promised lands of a functioning Gentoo arm64 LAMP. Some light querying of the Gentoo forums illuminated the path for me:

"In short, they mean forced, masked, or removed (also mentioned on man emerge)."

Yeah, okay, I failed to rtfm. Nevertheless, some digging in /usr/portage/profiles/arch/arm64/use.mask confirmed:

lobbes@pizrk003 ~ $ cat /usr/portage/profiles/arch/arm64/use.mask | grep 'mysql' -B 30

[ snip ]

# havent tested yet

ruby

ctypes-python

sasl

libproxy

pkcs11

bazaar

darcs

mercurial

kerberos

gpg

gnome

doc

emacs

gphoto2

afs

mono

lirc

lm_sensors

xemacs

rpm

scanner

zvbi

ldap

mysql

Aha! There's our culprit. The comment states it was masked due to it being untested1 so we are free to do some testing ourselves as far as I'm concerned.

The Gentoo wiki says the best way to unmask is to edit /etc/portage/profile/use.mask, so let's listen to it and add something like the following:

# Unmask mysql

-mysql

Now try emerging once more:

pizrk003 profile # emerge --ask dev-lang/php:5.6

* IMPORTANT: 4 news items need reading for repository 'gentoo'.

* Use eselect news read to view new items.

* Last emerge --sync was 66d 7h 27m 55s ago.

* IMPORTANT: config file '/etc/portage/package.use' needs updating.

* See the CONFIGURATION FILES and CONFIGURATION FILES UPDATE TOOLS

* sections of the emerge man page to learn how to update config files.

These are the packages that would be merged, in order:

Calculating dependencies... done!

[ebuild   R    ] dev-lang/php-5.6.35-r1  USE="mysql* -mysqli*"

Would you like to merge these packages? [Yes/No] yes

Why hello mysql!2 After the ebuild finished successfully I gave equery a query:

pizrk003 profile # equery u dev-lang/php-5.6.35-r1

[ Legend : U - final flag setting for installation]

[        : I - package is installed with flag     ]

[ Colors : set, unset                             ]

* Found these USE flags for dev-lang/php-5.6.35-r1:

U I

[...snip...]

- - libmysqlclient : Use libmyslclient driver for mysql, mysqli, PDO_Mysql (not recommended)

- - libressl       : Use dev-libs/libressl instead of dev-libs/openssl as SSL/TLS provider (ineffective with USE=-ssl), packages should not depend on this USE

flag

- - mhash          : Add support for the mhash library

+ + mysql          : Add mySQL Database support

Well hot diggity dog! One quick apache restart later confirms php mysql module is indeed functioning. Victory.

  1. vs. "known to cause massive breakages and anal fissures" []
  2. and goodbye mysqli []

Mein Kompendium: mp-wp

June 18th, 2018
This was originally posted March 24, 2018 before this blogs' temporary death on old VPS. Since then, hanbot has released a MP-WP genesis, and esthlos also posted a handy guide on erecting mp-wp on gentoo (which I consulted while re-erecting this blog on the pizarro rockchip).

This is my own compendium of fragmented notes I gathered together while erecting my own copy mp-wp. I started off by using hanbot's guide (archived), and as such I will not re-cover things already covered there. This is meant to be supplemental, ya dig?
---
Creating a mysql database, user, and assigning privileges:
This is assuming you already have mysql installed on the target box1 . First, log into mysql as root and take a look at what databases you currently have:
> mysql -u root -p
mysql> SHOW DATABASES;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
+--------------------+
3 rows in set (0.05 sec)
Now create a database and name it whatever you like:
mysql> CREATE DATABASE [Your database name];
Query OK, 1 row affected (0.00 sec)
mysql> SHOW DATABASES;
+--------------------+
| Database |
+--------------------+
| information_schema |
| [Your database name] |
| mysql |
| performance_schema |
+--------------------+
4 rows in set (0.00 sec)
Make that user!
mysql> CREATE USER '[Your user name]'@'localhost' IDENTIFIED BY '[Your password]';
Query OK, 0 rows affected (0.00 sec)
You can validate that it was created by checking the mysql.user table:
mysql> SELECT User FROM mysql.user;
+------------------+
| User |
+------------------+
| root |
| [Your user name] |
+------------------+
2 rows in set (0.00 sec)
Grant all privileges for your new database to your new user:
mysql> GRANT ALL PRIVILEGES ON [Your database name].* TO '[Your user name]'@'localhost' IDENTIFIED BY '[Your password]';
Query OK, 0 rows affected (0.05 sec)
Finally, flush privileges and exit:
mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)
mysql> EXIT;
Bye
Later, you will go through the install process2 and mp-wp will create all the tables in [Your database]. If you're like me, you'll want to see these newly created tables. Head back to mysql, and:
mysql> USE [Your database];
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> SHOW tables;
+-------------------------------------+
| Tables_in_[Your database] |
+-------------------------------------+
| [Your goddamned prefix]comments |
| [Your goddamned prefix]links |
| [Your goddamned prefix]options |
| [Your goddamned prefix]postmeta |
| [Your goddamned prefix]posts |
| [Your goddamned prefix]term_relationships |
| [Your goddamned prefix]term_taxonomy |
| [Your goddamned prefix]terms |
| [Your goddamned prefix]usermeta |
| [Your goddamned prefix]users |
+-------------------------------------+
10 rows in set (0.00 sec)
---
Configuring blog to use permalinks (Optional):
Disclaimer: I've only used apache thus far in my hackin' career, so no idea how this works with, say, nginx. This may or may not be of use to you.
To set your blog to use permalinks3, point yer browser to "[Your blog root url]/wp-admin/", log in, select the desired settings, and then make note of the code presented at the bottom of the page. Will look similar to the following:

RewriteEngine On
RewriteBase /[Your blog root dir]/
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /[Your blog root dir]/index.php [L]

Then you must create a .htaccess file in /var/www/[Your blog path] with the code mp-wp produced contained within.
I also had to4 enable apache2's mod_rewrite module and edit the config file in /etc/apache2/sites-available/ to AllowOverride for my blog directory.
Of course, these steps will vary depending on your system and root blog directory, so some research may be required.
Don't forget to restart apache once done configurin'!
---
Spiffy js-text selector:
For that nifty text-selection thing, first read this:
http://trilema.com/2015/that-spiffy-selection-thing/
Note: Remember to take your permalinks5 into account when specifying your path
Note2: I'm using my own 'forked' version of the "default" theme. This worked for me out-of-the-box, but if your theme has a different DOM than the "default", then you might find useful these tips spyked laid out in the logs
---
"Default" theme sidebar:
When hacking off of the "default" theme6 you will have to add7 "<?php get_sidebar(); ?>" to single.php to get the sidebar to appear on individual posts.
Note that some playing around with the css will most likely be necessary for things to not look retarded.
---
Custom rss feed preview:
This may be of no use to you if your entire site will be devoted to your blog. However, I wanted to shove my blog down a level in the hierarchy, and thus wanted my 'homepage' to remain 100% non-wordpress. While that part is easy enough, I still wanted to preview my posts on that homepage.
Luckily, I found a php RSS feed displayer-isor in the wild. Very short and easy to read and edit to your preferences (e.g. I removed the bit that previews the entire post); you can find the original here (archived).
  1. If you don't, it should not be difficult finding the proper install documentation for your system. []
  2. /wp-admin/install.php []
  3. e.g. "www.[Your blog root url]/2018/01/your-post-title" instead of "www.[Your blog root url]/?p=1234" []
  4. On Debian 7 []
  5. if you are using them []
  6. vs "classic" []
  7. take a look at page.php for an idea of where exactly []

lobbesbot Autobid Functionality

June 18th, 2018

This post was originally posted March 4, 2017 on old blawg layout


Following the suggestion from hanbot in #eulora, I set off to implement an auto-bidding feature for lobbesbot. Here are the basics of how it works:

!Qautobid [auction id (INT)] [max bid in coppers] [increment ammount in coppers]

Only accepted via pm. Enter your max bid amount in coppers for an auction id. Abbreviations like 'mn' and 'k' are accepted and are parsed accordingly. Increment amount specifies how much you will overbid up to your max auto-bid amount. E.g. If your max bid = 12mn, increment = 1mn, and the bot triggered your auto-bid against a bid of 6mn, your auto-bid would be 7mn. If it were instead responding to a bid of 11.5mn, your auto-bid would be your max of 12mn.

Using this command a consecutive time for a given auction id will update the values for that auction. Note: To set your values to 0 (remove your auto-bid), use the!Qauctionbidzero command detailed further down.

Each time someone 'manually' bids in channel, lobbesbot checks all outstanding auto-bids and resolves by overbidding either the manual bid in the channel or the second-highest max auto-bid outstanding for that auction id. The lobbesbot will reply with both the overbid amount and the amount of 'losing' auto-bids (0 if none).

In this example, 'dengus' and 2 other auto-bidders have previously entered their sekkrit max auto-bid amounts. I bid 12.1m coppers on auction 43. The lobbesbot goes through the computations and arrives at 'dengus' being the top-dog bidder in the scenario with a bid of 25mn, overbidding both my previous bid as well as the second-banana auto-bidder. Notice the '2 other auto-bid(s)' portion:

lobbes: !Qbid 43 12.1m

lobbes_devbot: AUCTION # 43: test Heard: 12.1mn from lobbes Ending: 2017-03-06 03:11:49 UTC (46 hours 59 mins)

lobbes_devbot: AUCTION # 43: AUTO-BID Heard: 25mn from dengus overbidding lobbes and 2 other auto-bid(s) Ending: 2017-03-06 03:11:49 UTC (46 hours 59 mins)

With regards to collisions between more than one auto-bid per auction: the auto-bidder with the highest max bid will always win. In cases where two auto-bidders have the same highest max bid, the winner will be decided on who entered their auto-bid first for that auction id. In other words, a priority system is employed. Note that the actual priority values are deliberately obfuscated from all bidders.

---

!Qautobidlist [takes no arguments]

Will only respond via pm, even if sent in channel. Lists all of the active auto-bids for the irc nick that sent the command.

E.g.

lobbes: !Qautobidlist

lobbes_devbot: Auto-bid for Auction # 45: Max Bid = 26mn coppers; Max Increment = 500k coppers

lobbes_devbot: Auto-bid for Auction # 46: Max Bid = 0 coppers; Max Increment = 0 coppers

---

!Qautobidzero [auction id (INT)]

Can be used to set all of your auto-bid amounts to 0 for a given auction id. Note: Your priority will be preserved. You can also use this to reserve your auto-bid priority for an auction before actually commiting to bidding anything.

E.g.

lobbes: !Qautobidzero 45

lobbes_devbot: Success: Autobid amounts have been set to 0 coppers for Auction # 45. Your priority has been preserved

---

Happy auto-bidding!