Searching from 2023-08-04 00:00:00 to 2023-08-04 23:59:59.999999.
Query completed in 1.56 seconds
[2023-08-04 02:57:24] <Lucifer_arma_> @delinquent: the consideration is reliability. I need a big innodb pool because it's one server serving a small parallel processing cluster. I've optimized the job queries pretty well, now the database is the bottleneck.
[2023-08-04 02:57:25] <armagetronbridge> 06irc:Lucifer_arma_| @delinquent: the consideration is reliability. I need a big innodb pool because it's one server serving a small parallel processing cluster. I've optimized the job queries pretty well, now the database is the bottleneck.
[2023-08-04 02:57:43] <Lucifer_arma_> Currently running on my little laptop, which is *not* good database hardware
[2023-08-04 02:57:43] <armagetronbridge> 06irc:Lucifer_arma_| Currently running on my little laptop, which is *not* good database hardware
[2023-08-04 02:59:10] <Lucifer_arma_> I can build a dedicated database right now on a raspi 4, 4GB, and I have a spare SSD I can use for it. So I'm weighing keeping the database on my laptop and throttling jobs to avoid overloading the database or moving it to a raspi that's only running the database.
[2023-08-04 02:59:10] <armagetronbridge> 06irc:Lucifer_arma_| I can build a dedicated database right now on a raspi 4, 4GB, and I have a spare SSD I can use for it. So I'm weighing keeping the database on my laptop and throttling jobs to avoid overloading the database or moving it to a raspi that's only running the database.
[2023-08-04 03:00:00] <Lucifer_arma_> That'll put two raspis as worker nodes, one as a database, and the scheduler running on my laptop. Four nodes in the cluster in total, with the laptop doing it's regular duty as my desktop PC as well
[2023-08-04 03:00:00] <armagetronbridge> 06irc:Lucifer_arma_| That'll put two raspis as worker nodes, one as a database, and the scheduler running on my laptop. Four nodes in the cluster in total, with the laptop doing it's regular duty as my desktop PC as well
[2023-08-04 03:01:30] <Lucifer_arma_> but the pi's, being four core, can easily be running 8 jobs simultaneously, all hitting the database at the same time using a python cursor.executemany call. So I need hardware that can handle that, and handle building out up to at least four total workers in the next month or so
[2023-08-04 03:01:30] <armagetronbridge> 06irc:Lucifer_arma_| but the pi's, being four core, can easily be running 8 jobs simultaneously, all hitting the database at the same time using a python cursor.executemany call. So I need hardware that can handle that, and handle building out up to at least four total workers in the next month or so
[2023-08-04 03:16:36] <-- Armanelgtron has quit (No Ping reply in 180 seconds.)
[2023-08-04 03:17:56] <-- Armanelgtron has quit (No Ping reply in 210 seconds.)
[2023-08-04 03:22:13] --> Armanelgtron has joined the channel
[2023-08-04 03:22:14] -!- tantalum.libera.chat set mode #armagetron +nt
[2023-08-04 03:22:14] -!- Channel #armagetron created on 2021-05-20 17:23:14 UTC
[2023-08-04 03:22:41] --> Armanelgtron has joined the channel
[2023-08-04 03:22:41] -!- Topic for #armagetron is "Armagetron Advanced | http://www.armagetronad.org/ | Welcome to IRC"
[2023-08-04 03:22:41] -!- Topic set by ChanServ!services@services.oftc.net on 2022-12-21 00:36:08 UTC
[2023-08-04 03:22:41] -!- weber.oftc.net set mode #armagetron +nt
[2023-08-04 03:22:41] -!- Channel #armagetron created on 2021-04-20 19:56:37 UTC
[2023-08-04 05:49:28] <-- Armanelgtron has quit (No Ping reply in 180 seconds.)
[2023-08-04 05:50:19] <-- Armanelgtron has quit (No Ping reply in 210 seconds.)
[2023-08-04 05:56:15] --> Armanelgtron has joined the channel
[2023-08-04 05:56:15] -!- lithium.libera.chat set mode #armagetron +nt
[2023-08-04 05:56:15] -!- Channel #armagetron created on 2021-05-20 17:23:14 UTC
[2023-08-04 05:56:39] --> Armanelgtron has joined the channel
[2023-08-04 05:56:39] -!- Topic for #armagetron is "Armagetron Advanced | http://www.armagetronad.org/ | Welcome to IRC"
[2023-08-04 05:56:39] -!- Topic set by ChanServ!services@services.oftc.net on 2022-12-21 00:36:08 UTC
[2023-08-04 05:56:39] -!- weber.oftc.net set mode #armagetron +nt
[2023-08-04 05:56:39] -!- Channel #armagetron created on 2021-04-20 19:56:37 UTC
[2023-08-04 07:02:19] --> monr0e has joined the channel
[2023-08-04 07:30:49] <armagetron-bridge> 08discord:delinquent| Lucifer_arma If you have large read/write ops, a larger InnoDB pool isn't a good move - especially on a RasPi which doesn't utilise ECC. A large InnoDB pool is more effective when you have a great deal of smaller ops, particularly as the pool won't be that large on a Pi to begin with, curtailing any effectiveness of the pool regardless. To put that into perspective, I have an ins <clipped message>
[2023-08-04 07:30:49] <armagetronbridge> 08discord:delinquent| Lucifer_arma If you have large read/write ops, a larger InnoDB pool isn't a good move - especially on a RasPi which doesn't utilise ECC. A large InnoDB pool is more effective when you have a great deal of smaller ops, particularly as the pool won't be that large on a Pi to begin with, curtailing any effectiveness of the pool regardless. To put that into perspective, I have an ins <clipped message>
[2023-08-04 07:30:49] <armagetronbridge> 08discord:delinquent| tance with a variable pool size, which usually sits at around 120Gb. Even that is not always enough to cache *everything*, and we have flash storage as our medium too. Ideally I'd have a more modular setup, but I'm already knocking at the limits of what power can be delivered.
[2023-08-04 07:30:50] <armagetron-bridge> 08discord:delinquent| tance with a variable pool size, which usually sits at around 120Gb. Even that is not always enough to cache *everything*, and we have flash storage as our medium too. Ideally I'd have a more modular setup, but I'm already knocking at the limits of what power can be delivered.
[2023-08-04 07:30:51] <armagetronbridge> 08discord:delinquent| If this is a smaller-scale project, it's far more effective to go with flash storage and a somewhat smaller pool - the host also needs memory for its own I/O, plus db backend, plus overhead. I'd give it 2Gb max, stick the SSD in it, and call it a day. Besides, a RasPi comes with other considerations - it claims, for example, to support Gigabit ethernet, and while that is technica <clipped message>
[2023-08-04 07:30:51] <armagetron-bridge> 08discord:delinquent| If this is a smaller-scale project, it's far more effective to go with flash storage and a somewhat smaller pool - the host also needs memory for its own I/O, plus db backend, plus overhead. I'd give it 2Gb max, stick the SSD in it, and call it a day. Besides, a RasPi comes with other considerations - it claims, for example, to support Gigabit ethernet, and while that is technica <clipped message>
[2023-08-04 07:30:52] <armagetronbridge> 08discord:delinquent| lly true, it can't necessarily maintain that bandwidth at a sustained rate. Not much point having a larger pool if you can never utilise it.
[2023-08-04 07:30:53] <armagetron-bridge> 08discord:delinquent| lly true, it can't necessarily maintain that bandwidth at a sustained rate. Not much point having a larger pool if you can never utilise it.
[2023-08-04 08:23:16] <-- Armanelgtron has quit (No Ping reply in 180 seconds.)
[2023-08-04 08:27:56] <-- Armanelgtron has quit (No Ping reply in 270 seconds.)
[2023-08-04 08:29:53] --> Armanelgtron has joined the channel
[2023-08-04 08:29:54] -!- cadmium.libera.chat set mode #armagetron +nt
[2023-08-04 08:29:54] -!- Channel #armagetron created on 2021-05-20 17:23:14 UTC
[2023-08-04 08:30:31] --> Armanelgtron has joined the channel
[2023-08-04 08:30:31] -!- Topic for #armagetron is "Armagetron Advanced | http://www.armagetronad.org/ | Welcome to IRC"
[2023-08-04 08:30:31] -!- Topic set by ChanServ!services@services.oftc.net on 2022-12-21 00:36:08 UTC
[2023-08-04 08:30:32] -!- weber.oftc.net set mode #armagetron +nt
[2023-08-04 08:30:32] -!- Channel #armagetron created on 2021-04-20 19:56:37 UTC
[2023-08-04 09:07:08] <-- Armanelgtron has quit (No Ping reply in 180 seconds.)
[2023-08-04 09:08:28] <-- Armanelgtron has quit (No Ping reply in 210 seconds.)
[2023-08-04 09:13:43] --> Armanelgtron has joined the channel
[2023-08-04 09:13:43] -!- Topic for #armagetron is "Armagetron Advanced | http://www.armagetronad.org/ | Welcome to IRC"
[2023-08-04 09:13:43] -!- Topic set by ChanServ!services@services.oftc.net on 2022-12-21 00:36:08 UTC
[2023-08-04 09:13:43] -!- weber.oftc.net set mode #armagetron +nt
[2023-08-04 09:13:43] -!- Channel #armagetron created on 2021-04-20 19:56:37 UTC
[2023-08-04 09:13:45] --> Armanelgtron has joined the channel
[2023-08-04 09:13:46] -!- lithium.libera.chat set mode #armagetron +nt
[2023-08-04 09:13:46] -!- Channel #armagetron created on 2021-05-20 17:23:14 UTC
[2023-08-04 09:17:35] <-- monr0e has quit (Quit: signalblue made me do this)
[2023-08-04 09:18:03] --> monr0e has joined the channel
[2023-08-04 09:31:59] <-- Armanelgtron has quit (No Ping reply in 180 seconds.)
[2023-08-04 09:33:49] <-- Armanelgtron has quit (No Ping reply in 180 seconds.)
[2023-08-04 09:40:15] --> Armanelgtron has joined the channel
[2023-08-04 09:40:16] -!- copper.libera.chat set mode #armagetron +nt
[2023-08-04 09:40:16] -!- Channel #armagetron created on 2021-05-20 17:23:14 UTC
[2023-08-04 09:40:52] --> Armanelgtron has joined the channel
[2023-08-04 09:40:52] -!- Topic for #armagetron is "Armagetron Advanced | http://www.armagetronad.org/ | Welcome to IRC"
[2023-08-04 09:40:52] -!- Topic set by ChanServ!services@services.oftc.net on 2022-12-21 00:36:08 UTC
[2023-08-04 09:40:54] -!- weber.oftc.net set mode #armagetron +nt
[2023-08-04 09:40:54] -!- Channel #armagetron created on 2021-04-20 19:56:37 UTC
[2023-08-04 09:50:01] <-- Armanelgtron has quit (No Ping reply in 180 seconds.)
[2023-08-04 09:54:51] <-- Armanelgtron has quit (No Ping reply in 270 seconds.)
[2023-08-04 09:55:45] --> Armanelgtron has joined the channel
[2023-08-04 09:55:45] -!- Topic for #armagetron is "Armagetron Advanced | http://www.armagetronad.org/ | Welcome to IRC"
[2023-08-04 09:55:45] -!- Topic set by ChanServ!services@services.oftc.net on 2022-12-21 00:36:08 UTC
[2023-08-04 09:55:46] -!- weber.oftc.net set mode #armagetron +nt
[2023-08-04 09:55:46] -!- Channel #armagetron created on 2021-04-20 19:56:37 UTC
[2023-08-04 09:56:49] --> Armanelgtron has joined the channel
[2023-08-04 09:56:50] -!- silver.libera.chat set mode #armagetron +nt
[2023-08-04 09:56:50] -!- Channel #armagetron created on 2021-05-20 17:23:14 UTC
[2023-08-04 10:11:12] <-- Armanelgtron has quit (No Ping reply in 180 seconds.)
[2023-08-04 10:12:32] <-- Armanelgtron has quit (No Ping reply in 210 seconds.)
[2023-08-04 10:17:09] --> Armanelgtron has joined the channel
[2023-08-04 10:17:09] -!- Topic for #armagetron is "Armagetron Advanced | http://www.armagetronad.org/ | Welcome to IRC"
[2023-08-04 10:17:09] -!- Topic set by ChanServ!services@services.oftc.net on 2022-12-21 00:36:08 UTC
[2023-08-04 10:17:10] -!- larich.oftc.net set mode #armagetron +nt
[2023-08-04 10:17:10] -!- Channel #armagetron created on 2021-04-20 19:56:37 UTC
[2023-08-04 10:17:11] --> Armanelgtron has joined the channel
[2023-08-04 10:17:12] -!- osmium.libera.chat set mode #armagetron +nt
[2023-08-04 10:17:12] -!- Channel #armagetron created on 2021-05-20 17:23:14 UTC
[2023-08-04 14:28:44] <-- monr0e has quit (Ping timeout: 244 seconds)
[2023-08-04 16:56:32] <-- Armanelgtron has quit (No Ping reply in 180 seconds.)
[2023-08-04 17:00:32] <-- Armanelgtron has quit (No Ping reply in 180 seconds.)
[2023-08-04 17:03:03] --> Armanelgtron has joined the channel
[2023-08-04 17:03:05] -!- tungsten.libera.chat set mode #armagetron +nt
[2023-08-04 17:03:05] -!- Channel #armagetron created on 2021-05-20 17:23:14 UTC
[2023-08-04 17:04:05] --> Armanelgtron has joined the channel
[2023-08-04 17:04:05] -!- Topic for #armagetron is "Armagetron Advanced | http://www.armagetronad.org/ | Welcome to IRC"
[2023-08-04 17:04:05] -!- Topic set by ChanServ!services@services.oftc.net on 2022-12-21 00:36:08 UTC
[2023-08-04 17:04:05] -!- larich.oftc.net set mode #armagetron +nt
[2023-08-04 17:04:05] -!- Channel #armagetron created on 2021-04-20 19:56:37 UTC
[2023-08-04 17:30:12] <-- Armanelgtron has quit (No Ping reply in 180 seconds.)
[2023-08-04 17:31:22] <-- Armanelgtron has quit (No Ping reply in 180 seconds.)
[2023-08-04 17:39:38] --> Armanelgtron has joined the channel
[2023-08-04 17:39:39] -!- silver.libera.chat set mode #armagetron +nt
[2023-08-04 17:39:39] -!- Channel #armagetron created on 2021-05-20 17:23:14 UTC
[2023-08-04 17:40:35] --> Armanelgtron has joined the channel
[2023-08-04 17:40:35] -!- Topic for #armagetron is "Armagetron Advanced | http://www.armagetronad.org/ | Welcome to IRC"
[2023-08-04 17:40:35] -!- Topic set by ChanServ!services@services.oftc.net on 2022-12-21 00:36:08 UTC
[2023-08-04 17:40:35] -!- larich.oftc.net set mode #armagetron +nt
[2023-08-04 17:40:35] -!- Channel #armagetron created on 2021-04-20 19:56:37 UTC
[2023-08-04 18:18:12] <-- Armanelgtron has quit (No Ping reply in 180 seconds.)
[2023-08-04 18:18:53] <-- Armanelgtron has quit (No Ping reply in 180 seconds.)
[2023-08-04 18:25:49] --> Armanelgtron has joined the channel
[2023-08-04 18:25:49] -!- Topic for #armagetron is "Armagetron Advanced | http://www.armagetronad.org/ | Welcome to IRC"
[2023-08-04 18:25:49] -!- Topic set by ChanServ!services@services.oftc.net on 2022-12-21 00:36:08 UTC
[2023-08-04 18:25:50] -!- reflection.oftc.net set mode #armagetron +nt
[2023-08-04 18:25:50] -!- Channel #armagetron created on 2021-04-20 19:56:37 UTC
[2023-08-04 18:25:56] --> Armanelgtron has joined the channel
[2023-08-04 18:25:57] -!- tungsten.libera.chat set mode #armagetron +nt
[2023-08-04 18:25:57] -!- Channel #armagetron created on 2021-05-20 17:23:14 UTC
[2023-08-04 20:58:46] --> monr0e has joined the channel
[2023-08-04 22:31:14] <-- monr0e has quit (Ping timeout: 246 seconds)
[2023-08-04 22:46:59] <Lucifer_arma_> @delinquent: Right now, they're only "large" because it's updating almost from scratch. After that, the inserts will be small. Right now, the database is on this: Intel(R) Pentium(R) Silver N5030 CPU @ 1.10GHz
[2023-08-04 22:46:59] <armagetronbridge> 06irc:Lucifer_arma_| @delinquent: Right now, they're only "large" because it's updating almost from scratch. After that, the inserts will be small. Right now, the database is on this: Intel(R) Pentium(R) Silver N5030 CPU @ 1.10GHz
[2023-08-04 22:48:11] <Lucifer_arma_> it's also an SSD, because that's what it came with. If I move the database to the other pi, I'd be willing to buy an inexpensive hard drive if it's more reliable and/or faster (but just as reliable). I'm not yet willing to get too crazy with optimizing the database when I still have code to work on.
[2023-08-04 22:48:12] <armagetronbridge> 06irc:Lucifer_arma_| it's also an SSD, because that's what it came with. If I move the database to the other pi, I'd be willing to buy an inexpensive hard drive if it's more reliable and/or faster (but just as reliable). I'm not yet willing to get too crazy with optimizing the database when I still have code to work on.
[2023-08-04 22:49:02] <Lucifer_arma_> I *can* slow down job execution, and there's a few places I'm hitting the database unnecessarily often, that sort of stuff.
[2023-08-04 22:49:02] <armagetronbridge> 06irc:Lucifer_arma_| I *can* slow down job execution, and there's a few places I'm hitting the database unnecessarily often, that sort of stuff.
[2023-08-04 22:49:43] <Lucifer_arma_> I don't have a read on how much bandwidth the pi's are actually using. The wifi is also gigabit, and supposedly, the problems the pi 3 had are fixed, so supposedly the pi 4 has full gigabit capabilities
[2023-08-04 22:49:43] <armagetronbridge> 06irc:Lucifer_arma_| I don't have a read on how much bandwidth the pi's are actually using. The wifi is also gigabit, and supposedly, the problems the pi 3 had are fixed, so supposedly the pi 4 has full gigabit capabilities
[2023-08-04 22:51:43] <Lucifer_arma_> So, what I'm getting from this conversation is that I should leave the database as-is (even with the increased innodb pool that I already did) and focus on the cluster software. If the database continues to get bogged down, throttle down job execution, and later, when this damn thing has made me some money, build a good database server that can handle everything
[2023-08-04 22:51:45] <armagetronbridge> 06irc:Lucifer_arma_| So, what I'm getting from this conversation is that I should leave the database as-is (even with the increased innodb pool that I already did) and focus on the cluster software. If the database continues to get bogged down, throttle down job execution, and later, when this damn thing has made me some money, build a good database server that can handle everything
[2023-08-04 22:56:25] <-- Armanelgtron has quit (No Ping reply in 180 seconds.)
[2023-08-04 22:57:45] <-- Armanelgtron has quit (No Ping reply in 210 seconds.)
[2023-08-04 23:00:55] --> Armanelgtron has joined the channel
[2023-08-04 23:00:55] -!- Topic for #armagetron is "Armagetron Advanced | http://www.armagetronad.org/ | Welcome to IRC"
[2023-08-04 23:00:55] -!- Topic set by ChanServ!services@services.oftc.net on 2022-12-21 00:36:08 UTC
[2023-08-04 23:00:55] -!- reflection.oftc.net set mode #armagetron +nt
[2023-08-04 23:00:55] -!- Channel #armagetron created on 2021-04-20 19:56:37 UTC
[2023-08-04 23:01:57] --> Armanelgtron has joined the channel
[2023-08-04 23:01:58] -!- tungsten.libera.chat set mode #armagetron +nt
[2023-08-04 23:01:59] -!- Channel #armagetron created on 2021-05-20 17:23:14 UTC
View entire month
DISCLAIMER: These logs of public chat may contain some content which may not be appropriate for all audiences. Use at your own risk.
Logs from 2006-2009 pulled from wrtlprnft
Format changes at: 2015-08-25, 2017-02-20, and 2020-03-23. Times (2015 and later) should be Eastern.
• Website was created by nelg
• Design:
» Layout is loosely based on an old design by Durf.
» Icons on the tools page were originally by Royce, new icons added will try to follow some of the design language.
» Various other icon selections include icons from Crystal, crystalsvg, oxygen, and breeze icon themes.
» Other icons: fontawesome
is a site that has tools, maps, and other various items. It orignally only existed as a server control panel, but I slowly began adding stuff such as a homepage and tools.
Gridder
.