RPiAqua (Raspberry Pi Light Controller - In Progress) - Page 2 - The Planted Tank Forum
 6Likes
Reply
 
LinkBack Thread Tools Display Modes
post #16 of 106 (permalink) Old 07-24-2018, 10:53 PM
Algae Grower
 
PTrader: (1/100%)
Join Date: Nov 2017
Location: Burlington, VT
Posts: 11
Quote:
Originally Posted by Wobblebonk View Post
I'm tired of explaining to a new rep (from other companies that license a product from my work) how to update the licenses that expire every year.
It's simple. You explain that enterprise support is a reasonable monthly fee which includes making sure these sorts of unfortunate annoyances are dealt with.
staticicarus is offline  
Sponsored Links
Advertisement
 
post #17 of 106 (permalink) Old 07-25-2018, 05:11 AM Thread Starter
Planted Member
 
Kaiede's Avatar
 
PTrader: (0/0%)
Join Date: Sep 2017
Location: Seattle, WA
Posts: 247
Tonight, I got the scheduler implemented, along with the schedule preview tool. Not terribly happy with the state of the code, as it isn't as clean as I normally like to keep it. But I did manage to slip in a couple tweaks I wasn't originally intending to tonight:
  • Gamma Support (fixed at 1.8 for now)
  • Configurable PWM frequency (Supports any multiple of 480 Hz, up to 2880 Hz for the built-in PWM hardware)

Interesting thing is that I noticed that neither the Bluefish or Fluval does any sort of gamma correction. Their behaviors are completely linear. Neat.

I've kicked off the schedule, and hooked the thing up to one of my tanks to see if it actually works. Unfortunately, there are still bugs with it not moving onto the next part of the ramp once one finishes. I think I have a rough idea what's going on, and I'll go ahead and make the fix tomorrow.

I've also moved it out from where I host my private repos to GitHub. So it is available to poke at. But there aren't any instructions yet. I'll be looking at that once I stabilize things a bit and fix the issues that remain.

https://github.com/Kaiede/RPiLight

Quote:
Originally Posted by f-fish View Post
Think the approach of doing one thing well and keeping it sensible and focused plays well when creating building blocks. You say just automating lights, but it is more than just on / off so well worth the effort. Keen to understand, are you or do you plan on running multiple tanks or is your current need to only do this for 1 tank? Reason I ask, I have found that monitoring before automation plays a bigger drive when you have multiple tanks and need to have that single view of what is actually going on. Having looked at several all-in-one aquarium projects, I have also opted for purpose build solution blocks that can be operated centrally but that actually function autonomously for that one function.

Looking forward to seeing it on github ...
In my case, I have two tanks, and if I could put them near each other, and use a single controller, I would. But they aren't, so the plan is to use two controllers.

It would be nice to have central control, and deploying individual config/schedule JSON blobs out to each RPi. But seeing as having the front-end is kinda far out there at the moment, I'm not putting any effort into thinking about this right now.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Kaiede is online now  
post #18 of 106 (permalink) Old 07-26-2018, 01:42 AM Thread Starter
Planted Member
 
Kaiede's Avatar
 
PTrader: (0/0%)
Join Date: Sep 2017
Location: Seattle, WA
Posts: 247
And now for the fun bit. I'm pretty confident that I've fixed the issue with scheduling. I've hooked up the Twinstar 600E I've got to it and will see how the controller works. I'll be keeping an eye on it for the next few days to make sure things are reliable. But unless there's remaining bugs, it can handle a 24-hour ramp just fine. It already handles being started in the middle of the schedule, so no real worries there.

I've got a couple more tasks to do, which are tracked on GitHub. This is the remaining work I consider important enough for the first release. But right now it should be usable, but has a couple dependencies that need to get installed before it works on a fresh Raspbian install.

If there's interest, I can put together a small demo video this weekend of it driving a Fluval Vista LED strip.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Kaiede is online now  
 
post #19 of 106 (permalink) Old 07-27-2018, 01:32 PM
Planted Tank Guru
 
PTrader: (0/0%)
Join Date: Jun 2013
Location: WI
Posts: 10,929
Always interested....

"A man with a watch knows what time it is. A man with two watches is never sure."
jeffkrol is online now  
post #20 of 106 (permalink) Old 07-27-2018, 05:12 PM Thread Starter
Planted Member
 
Kaiede's Avatar
 
PTrader: (0/0%)
Join Date: Sep 2017
Location: Seattle, WA
Posts: 247
Some good news and some weird news.

The good news is that after fixing an off-by-one bug (plus some typos) in my PWM controller code, the adafruit PWM expansion board works perfectly for driving the MOSFET switches. Thank you, PCA9685 datasheet.

The weird news is that I also got a Pi Zero W in, and the performance I'm seeing is a little bit concerning, but not a huge problem in the short term. The Pi 3B with apscheduler was able to stay under 30% CPU while updating the PWM duty cycle 100 times a second. Excessive, but the idea was to have the rate high for quick ramps, and low for longer ones, using "just enough" CPU. But that setting on the Pi Zero causes it to max out the CPU just running the loop. Oops.

Short term, I can keep CPU usage below 30% just by changing the cap to something more reasonable.

Longer term, I may have to rethink the use of apscheduler. It's overhead in my case is enormous on the Pi Zero. It adds seconds of boot time when running any of the scripts. Starting a scheduler lags for a couple seconds before it starts handling jobs. At 24 updates a second, it consumes nearly 30% CPU just writing out "datetime.now()" in a loop. Doing the same myself uses 2.5-6%. It's perfectly fine on the 3B, though, which meant I didn't run into this until I had the Zero in hand. And it's not a python 2 vs 3 issue either. As I said, it's weird.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Kaiede is online now  
post #21 of 106 (permalink) Old 07-28-2018, 05:50 PM Thread Starter
Planted Member
 
Kaiede's Avatar
 
PTrader: (0/0%)
Join Date: Sep 2017
Location: Seattle, WA
Posts: 247
So, I decided to jettison apscheduler entirely. I was still getting some race conditions with it, since apscheduler apparently doesn't guarantee much in the way of ordering of tasks if two of them happen to have the same date/time for firing. Great. So I figured since I was also seeing perf issues, it was going to about as complex to write a custom (but simple) event loop where I can guarantee ordering and behavior, than try to fix the race conditions. It also opened up a couple more ways to further reduce CPU use during flat parts of the schedule.

I did some experiments with coroutines just for grins, and the perf was not all that much better than apscheduler. Better, but not quite as performant as my solution. I don't think I can quite hold to single digit CPU use during fast ramps, but its under 15% worst case, which is still a huge improvement.

While watching the log stream from my prototype scheduler this morning, I put together some documentation. It may be missing some things, and it can probably be massaged a bit to be easier to follow, but it is a full brain dump on how things work. That's available on the GitHub repo.

I can't quite put together a video yet. Mostly because all my MOSFET switches are wired up and in use. I have more showing up today, and hopefully I can integrate the new scheduler (it's been looking really good so far) in short order. Mostly I just broke the test/preview scripts, which relied on behavior I haven't duplicated in the new scheduler (didn't need it). I think tomorrow I can put together a quick demo video showing the quick ramp behavior and previewing a schedule.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Kaiede is online now  
post #22 of 106 (permalink) Old 07-29-2018, 01:27 AM Thread Starter
Planted Member
 
Kaiede's Avatar
 
PTrader: (0/0%)
Join Date: Sep 2017
Location: Seattle, WA
Posts: 247
It's a simple demo, but I threw it together. The test ramp intentionally "hiccups" in the middle of the second ramp, to test overlapping segments. The preview ramp is a simple schedule (the one included in the examples folder) that includes a "morning" and "evening" period around a bright period. Unfortunately, it's also a bit difficult to capture a test rig like this without blowing it out from all the light.

YouTube - RPiLight Demo


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Kaiede is online now  
post #23 of 106 (permalink) Old 07-30-2018, 05:21 AM
Planted Tank Guru
 
PTrader: (0/0%)
Join Date: Jun 2013
Location: WI
Posts: 10,929
Thanks, I'd say things are looking good, but I only understand about a tenth of what you say...LOL..

"A man with a watch knows what time it is. A man with two watches is never sure."
jeffkrol is online now  
post #24 of 106 (permalink) Old 07-30-2018, 06:08 AM Thread Starter
Planted Member
 
Kaiede's Avatar
 
PTrader: (0/0%)
Join Date: Sep 2017
Location: Seattle, WA
Posts: 247
RPiAqua (Raspberry Pi Light Controller - In Progress)

Quote:
Originally Posted by jeffkrol View Post
Thanks, I'd say things are looking good, but I only understand about a tenth of what you say...LOL..

I sometimes forget tech-savvy doesnít always mean engineer, or the same kind of engineer. This sort of project is relaxing compared to the complexity of projects I do for work.

Anyhow, I tagged v0.1 for release this morning before having to tag v0.1.1. There was a nasty bug that showed up when the tool was started early in the morning.

But it does have a release that anyone can try if they really want.

Itís not exactly friendly to those who arenít tech-savvy enough to muck with Linux and do a bit of wiring, but itís been running my Twinstar 600E for a couple days straight now. Itís handling that core functionality about as well as a TC-420. Just more DIY, and less proprietary software involved. I wonít miss PLed.

Since Iíve been burning the candle at both ends this week, I am going to take a break and see if any bugs show up while Iím not making changes. Next weekend Iíll see where things are and start looking at the next set of additions.

At first, I think Iíll do some smaller, but useful tweaks before trying to tackle a larger feature. Things Iíd like to be able to configure without code changes, like custom gamma, or being able to tell the software when a channel shuts off. The Twinstar starts behaving oddly when at about 0.2%, with the red and green LEDs turning off before the blue and white do. Itíd be great to have the controller account for that sort of thing if told about it.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Kaiede is online now  
post #25 of 106 (permalink) Old 07-30-2018, 06:39 AM
Planted Tank Enthusiast
 
doylecolmdoyle's Avatar
 
PTrader: (0/0%)
Join Date: Sep 2015
Location: Western Australia
Posts: 623
Cool project, FYI you can run PLED on osx via Wine, it works like a charm!


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
doylecolmdoyle is offline  
post #26 of 106 (permalink) Old 07-30-2018, 01:14 PM Thread Starter
Planted Member
 
Kaiede's Avatar
 
PTrader: (0/0%)
Join Date: Sep 2017
Location: Seattle, WA
Posts: 247
Quote:
Originally Posted by doylecolmdoyle View Post
Cool project, FYI you can run PLED on osx via Wine, it works like a charm!

You arenít even the first person to tell me that in this thread. I had problems with PLed. It stopped working because of some update at some point. Already spent too much time on that rabbit hole.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Kaiede is online now  
post #27 of 106 (permalink) Old 08-02-2018, 05:00 PM Thread Starter
Planted Member
 
Kaiede's Avatar
 
PTrader: (0/0%)
Join Date: Sep 2017
Location: Seattle, WA
Posts: 247
Okay, now folks can call me crazy. I just about rewrote the v0.1 from scratch solely to get better performance.

The python version I wrote isn't terribly great when it comes to CPU. A single on-board PWM channel was enough to run the CPU at full load if I updated frequently enough. By updating less frequently, and also by replacing the scheduler library with a custom loop, I was able to get the CPU use down to about 14% for that single channel. But what about when using the Adafruit PWM board, which has 16 channels on it? Those extra channels cost CPU time, as does the communication with the board. Maybe not 16x, but it would still rise noticeably. I just wasn't really happy with where this was going. That 14% didn't include the extra 5-9% the GPIO daemon was using in the background for things that didn't benefit the project.

So when I noticed that Swift works on these boards, I figured I'd do some prototyping, just in case. There's a couple good things about Swift in this space that would make it easier to do this sort of time-based programming, I'm familiar with it, and bootstrapping projects with it is faster than with C/C++ (which was my next choice).

The good news is that the prototyping was very promising. That single channel test was looking really good, and the numbers for 24 updates per second looked like:
  • Python: 28% CPU
  • Python (Optimized): 14% CPU
  • Swift: 1.3% CPU

Because of the huge improvement, I decided to do some real stress testing. Up the rate to 100 updates per second, and look at different channel counts:
  • On-Board (1 Ch): 4.9%
  • On-Board (2 Ch): 6.8%
  • Adafruit PWM (1 Ch): 7.5%
  • Adafruit PWM (2 Ch): 12%
  • Adafruit PWM (4 Ch): 19%
  • Adafruit PWM (16 Ch): 26%

The end result is that I can update 16 channels, 4 times as frequently as before, and still use a little less CPU to do it. These numbers in general are much closer to what I was hoping to be able to achieve. I can probably shrink the gap between the On-Board and the Adafruit PWM a bit, since the communication could be made more efficient, but that would need a bit of work. For now, since these sort of bursts of rapid updates should be rare, this is a sort of worst-case situation, and I'm okay with 16 channels needing that much CPU briefly. At least for now.

But it does mean I've spent a couple days this week rewriting everything in Swift, and I'm not quite done yet before I'm back at parity with v0.1. What's left is fairly small, routine stuff though.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Kaiede is online now  
post #28 of 106 (permalink) Old 08-06-2018, 04:32 AM Thread Starter
Planted Member
 
Kaiede's Avatar
 
PTrader: (0/0%)
Join Date: Sep 2017
Location: Seattle, WA
Posts: 247
I just posted 0.1.6 to GitHub. This is the version re-written in Swift. It does have a better install process, and the config files are almost entirely backwards compatible. I did merge the config.json and schedule.json into a single file though, and I renamed the channels to be a bit shorter. Figure it'd be easier to break configs for the 2 people even taking a look at this now than later.

It uses a decent chunk of RAM because of libraries (20MB), but that could be a whole lot worse. It's under 5% of the available RAM on the Pi, and won't grow all that much as I add new features in.

Quote:
Originally Posted by Wobblebonk View Post
Fun... my dumbass wanted to save work by getting a premade controller and somehow now I'm rewriting their firmware for them. I probably should have just built one but hindsight and all that...
Since you are also messing around with this. Would you mind if I picked your brain around gamma?

The thing I did, and I'm starting to think may have been boneheaded, was apply gamma to the entire brightness range. So if I ask for 50% brightness, I am actually asking for more like 30% output due to applying gamma correction. But this actually makes certain things harder, despite making the ramp look nicer to my eyes.

Say I've got a light that produces 100 PAR at the substrate. If I put the PWM at 50% duty cycle, I'm going to read 50 PAR at the substrate. This is all well and good, and how someone in the hobby concerned about PAR will be thinking. So, knowing they want their photoperiod at 50 PAR, they put in 50% brightness. Now because my code is applying gamma correction, 50% brightness becomes ~30% intensity when using a gamma of 1.8. This means that they are actually getting closer to 30 PAR at the substrate, not the 50 they were expecting. This is not a great experience. But it's also possible I'm mis-applying gamma correction here, compared to computer displays.

So here's my question. Does this track with your implementation/thinking?

I'm wondering if I should make one of the following changes:
  • Make the schedule work in intensity instead of brightness, or even let the user specify which they want. So folks aiming at a particular PAR use intensity, folks who want a particular apparent brightness use that. I can still use apparent brightness when calculating the ramp if we want a ramp that looks linear to the human eye.
  • Abandon gamma if the goal is a curve that rises quickly and early when turning on, or tapers off more rapidly at the end. Because the math is being done in floating point, it's easier to do something like 0.4% intensity to get that full moon effect.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Kaiede is online now  
post #29 of 106 (permalink) Old 08-06-2018, 03:04 PM
Planted Tank Enthusiast
 
PTrader: (0/0%)
Join Date: Feb 2018
Location: Maryland
Posts: 683
OSX IS THE DEVIL.

but uh, I actually took out the cie1931 thing for the hurricanex because the lut took up too much memory (there's 2kb ram... hah) and I could have moved it to program space but they requested to show %s and it's a neat feature for manual modes and such when you're playing around but when setting it up I think actual intensity makes more sense.

When there is more memory available I may make it like a user option in the control gui (iphone / android app?) but it's like a neat feature that is less practical most of the time.
Wobblebonk is online now  
post #30 of 106 (permalink) Old 08-06-2018, 03:38 PM
Planted Tank Guru
 
PTrader: (0/0%)
Join Date: Jun 2013
Location: WI
Posts: 10,929
Quote:
Originally Posted by Kaiede View Post
I just posted 0.1.6 to GitHub. This is the version re-written in Swift. It does have a better install process, and the config files are almost entirely backwards compatible. I did merge the config.json and schedule.json into a single file though, and I renamed the channels to be a bit shorter. Figure it'd be easier to break configs for the 2 people even taking a look at this now than later.
Funny..




Quote:
Originally Posted by Kaiede View Post
I'm wondering if I should make one of the following changes:
  • Make the schedule work in intensity instead of brightness, or even let the user specify which they want. So folks aiming at a particular PAR use intensity, folks who want a particular apparent brightness use that. I can still use apparent brightness when calculating the ramp if we want a ramp that looks linear to the human eye.
  • Abandon gamma if the goal is a curve that rises quickly and early when turning on, or tapers off more rapidly at the end. Because the math is being done in floating point, it's easier to do something like 0.4% intensity to get that full moon effect.
Initial choice would be the choice..
An option of gamma vs linear is brilliant..
Number of separate curves or just one is debatable..If calculated.. not an issue if LUT then ??


Sometimes there is a fine line between too much and too little user adj...


Gamma dimming is becoming more "popular" in industry.. so there is that new car smell effect..
coupled w/ the higher Hz is better getting some traction...
1) https://www.gammalighttherapy.com/bl...-hz-led-dimmer
2) https://www.lrc.rpi.edu/programs/sol...ds/dimming.asp



Arguable that it is "needed" (gamma dimming) in our use but certainly "proper"..

"A man with a watch knows what time it is. A man with two watches is never sure."
jeffkrol is online now  
Reply

Tags
None

Quick Reply
Message:
Options

Register Now



In order to be able to post messages on the The Planted Tank Forum forums, you must first register.
Please enter your desired user name, your email address and other required details in the form below.

User Name:
Password
Please enter a password for your user account. Note that passwords are case-sensitive.

Password:


Confirm Password:
Email Address
Please enter a valid email address for yourself.

Email Address:
OR

Log-in










Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page
Display Modes
Linear Mode Linear Mode



Posting Rules  
You may post new threads
You may post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On

 
For the best viewing experience please update your browser to Google Chrome