I just wanted to announce, that I have started integrating a GRBL-based stage into the beautiful openflexure server system. There are many things to be done, but so far it works quite ok. It works with the standard Arduino + CNC Shield v3. The current forked version can be found here:
If you have anything you would like to see in there, I would be happy to take this as an opportunity to improve the code! (I’m not a programmer unfortunately
Thanks a lot for the development of this great tool!!
Really cool. I do worry about doing large amounts of development where the pipelines fail and the build system has been totally changed. I see that this work is being done in you master branch but that the first thing that is done is removing all of the poetry files and creating a setup.py based install interface. I also see that the camera work and the GRBL work is done in this same master branch.
As both of these will require significant testing before they can be merged it would be good if they were each in feature branches, and used poetry so that all of the tests we do on GitLab can run.
I am not saying this to discourage you. I think this work is fantastic. I just don’t want you to pour in tonnes of work in a way that we cannot easily merge. Happy to have a call about branches, and tests etc if this will help. Personally I also know setup.py not poetry so I can’t explain it, but Joel has good distribution reasons for using poetry so it would be good to get this to work on that.
I’d just like to second this. Our use of Poetry in particular has been thought though extensively, and is related to our strict requirement for both distributing with pinned dependencies, while also allowing developers to easily upgrade these dependencies.
Setup.py is really a mechanism for distributing libraries, not applications. The closest thing to what we need would be a requirements.txt file, however that comes with its own extensive limitations, so the choice for us ends up being between Pipenv and Poetry, of which Poetry was found to be the least awful.
Regarding the CI tests, these are fine to leave failing while you’re in the thick of development, but as Julian mentioned we really need everything to be passing before we can merge anything in. This is especially important for the MyPy static type checking, which helps us identify potential bugs very early into development.
If you have a look around the codebase you’ll see a lot of type annotations around, which should be a good reference. Once you’re reasonably happy with the code, you’ll need to add these kinds of type hints to your new sections. They’re fairly straightforward but help a lot with testing
Actually one other useful thing! If you run poetry run poe check you can test if the CI will pass your Python code. It runs all of our analysis locally
Hey, thanks for the feedback! I totally copy that.
As said, I’m really not a programmer and I try to fight myself through the code. So far: I survived
This git-branching thing is very handy I agree, perhaps it would indeed be a nice idea to schedule a meeting to talk about the best possible path to make it not only useful for me. I will try to use the openflexure server for a plate reader and eventually also for the UC2-stuff. The code is beautiful, now there is an elephant coming and destroys it.
Regarding Poetry: I agree, that’s much nicer. But since on the Jetson I wanted to get it working first to see if the GUI works in general, I did the fast route of using pip since I’m not familar with poetry at all. I think reverting back to poetry would not be an issue since I didn’t change anything.
Regarding the base system: In fact, I have tried to keep changes as small as possible. I guess I just had to add the entries for the GRBL stage and the new camera. Everything else is still subclased from their base versions. Only 2 modifications in the GUI. The reason why I did that instead of using extensions was, that I wanted ot use the nice scanning features inside the GUI. Which now works beautifully
Excellent - as someone who also works on the OFM software but hasn’t been as immersed as Joel, I appreciate the frustration of spending longer setting up the build environment than actually writing the new code! I think that’s somewhat inevitable once you start running projects with more automated quality checking, and it does pay off eventually.
Talking to Julian about the best git strategy is probably helpful - I’m aware he’s spent a lot of time recently merging people’s changes in while fixing various things in their history (pulling in the changes is easy, but preserving proper credit in the git history, so we can see who made which contribution, is much harder). If I was doing it, I’d probably suggest that the easiest thing to do is to save what you’ve done so far, go back to a clean copy (of the master branch, with poetry etc.), and then move your changes back in. Then, you can make a merge request that’s much more likely to go in smoothly - and it will all show up properly in the metadata. As Julian says, probably one merge for the stage and one for the camera would be clearest.
Having said that, Julian is much deeper in the git workflow than me at the moment - so I’d take his advice over mine!
I think something like that is right. Ugly dev code is fine, it if gets tidied later. However changes to core things, even if they get changed back, can cause numerous history conflicts that can take hours to unravel on merge.
Probably best to checkout the last commit before these changes started
Make a working dev environment on you machine
Make a feature branch for the camera and copy in those changes. - Try to minimise changes to existing code where possible
Make a feature branch for the motor stuff and copy in those changes - Again try to minimise changes to existing code where possible
If you need to use things with both motor and camera stuff make a local branch which is a merge of both motor and camera.
With this workflow it should be possible the keep the camera and motor code separate which will allow us to review them more easily. It should allow you to test both of them together. And it should allow the automated tests on GitLab to run which will let you know about problems in the code.
I also share Richard’s frustration with the time it takes to set up build environments. But as the project has reached the stage where automated testing is essential it is inevitable. The other thing that becomes frustrating is that we are getting towards the point where we spend 1 hour getting it to work, 2 hours making it tidy, 1 more hour documenting, and a then another 3 hours writing tests, and fixing edge cases in the code. Not saying you need to be doing that all now, but it will need to be done eventually if we are to get things in the released code base. Turns out software dev is much harder than hacking code together.
All this being said, if all you want for now is to have something that works for you, then do it however you want. I am mostly bringing this up because I don’t want you to do weeks of coding that you hope to be merged into the project only to find that we cannot merge it without significant re-writing.
So, basicly openflexure server should work with delta cnc as is? Can I take delta 3d printer frame, mix it with low magnification microscope (x2-x10) and use it with openflexure server?
GRBL is really interesting. It is relatevely easy to motorise old light microscopes and there is a whole bunch of tasks one can use them for… How it is going?
Hey, should work in principle!
So I managed to get around some of the annoying bits of GRBL recently. It works, but its hacky. I’m not a software engineer. But the only thing you need to do in the end is add the GRBL and GRLBBOARD.py to the stage folder and add a new stage to the microscope.py.
You can have a look here:
@j.stirling, I should perhaps create a seperate branch, force pull your master and simply replace the stage-bit and add the line in the loading script. The rest is in-line with your code. So far it is just lots of dirty hacking. Sorry.
Right now it works on a Nvidia Jetson Nano and Raspberry Pi.
Don’t branch from yours and force pull ours as this may create and entangled history that will be much harder to untangle later. General rule of thumb is that any command in Git that involves “force” or any command that back edits the history like rebase, should only be done with exceptional caution if you really understand the exact consequences.
Probably the best thing for now is to keep yours as it is and carry on playing around with making it work. At some point we can look at how we bring the two in sync. If the solution is a bit hacking then probably someone will have to cherry pick some of your commits and then tidy them up. This is probably the most efficient way for you to keep doing your thing, and for us to eventually support it once someone has the time to implement it robustly, but to still give you credit in the history when they cherry pick your commits.
Not saying you need to be doing that all now, but it will need to be done eventually if we are to get things in the released code base. Turns out software dev is much harder than hacking code together.