The Combine Forum banner

1 - 14 of 14 Posts

·
Registered
Joined
·
87 Posts
Discussion Starter #1 (Edited)
This is a topic lining out a specific idea that we (two farmers, farm advisor, tutor) are hoping to work on in the near future together with thesis students. I find it so promising that I want to share it with you, the AOG community. Free to build on and share ideas!

The context is that we are organic farmers who use a hoe in the front of the tractor. An example of this one-man operation: https://twitter.com/kadacode/status/1128024872263794688 The hoe blade tips leave 8 or so centimeters of open space for the crop row to pass, so imagine the concentration needed as a driver. We have RTK auto steer systems but unfortunately we can't use them with this operation because of build-up of small errors in the tractor-planter linkage, draft and so on, risking crop damage.
State-of-the-art solves this by attaching the hoe in the back together with a camera which can see the effective crop rows and compensates deviation from the crop line with a sideshift or steering disks. Example: https://twitter.com/search?q=rtk disc&src=typd

But we don't want that... We want to keep the hoe in front where we see it and attach something useful in the back. The initial idea of my colleague farmer is that we take this exact camera and attach it on the hoe in the front, and that deviation of the crop line is compensated by using the autosteer which is already on our tractor! So instead of RTK-GPS being the position source, it is a front camera, seeing the real crop. Sounds simple, but the companies that we contacted and are involved in the state-of-the-art are not really up for it. It seems they either have the know-how of camera row guidance or that of tractor autosteer, but never both. So we are at a point where we want to experiment ourselves.

AgOpenGPS, if I am not mistaking, essentially takes a position, a heading and a steering angle et voilà, you have your tractor driving along a line. Now here is the manual of a stereo-vision camera system produced by IFM and proposed to us by a technology tutor: https://www.ifm.com/mounting/706424UK.pdf
On pages 129-135 you can find the line guidance feature. The software can spit out almost continuously the offset of position to the crop line and the line angle to the driving direction. So the position and heading of the vehicle are known and theoretically, this information stream could be used in AgOpenGPS to autosteer the tractor along the effective crop rows.

This specific stereo vision camera costs around 1500 EUR and it is primarily built for following swaths and crop edges, so yet to be confirmed it works with small crop rows. Similar state of the art camera systems by for ex. Claas sell for 10x that price and are proven to work with the kind of operation we have in mind. We are now hoping for thesis students to apply for proposals we made on this very idea of integrating camera crop guidance with (open source) autosteer systems. Both technical and more theoretical (software, code).

A lot of questions remain, yet I think it would sooo work. I wish I had a better understanding of coding so we could get such a camera somewhere and start to experiment...
 

·
Registered
Joined
·
1,207 Posts
I was looking into the swath following last night for haying season, crazy expensive systems. We are both in a similar boat for guidance, gps is not the best option for me. I wonder if a infrared sensor would work like in the line following robots. Not sure if there would be enough difference in contrast though.
http://files.aust-eee-2211-project.webnode.com/200000093-5d4975e40a/Line Following Robot With Color Detection.pdf
I also came across something like this.
https://www.instructables.com/id/Line-Follower-Robot-Using-Camera-Shield-in-1Sheeld/
 

·
Registered
Joined
·
5,823 Posts
Very cool project - so specialized. OpenCV may also be an option depending on programming skill.

So is this the kind of system you are referring to? The control is insanely accurate.

 

·
Registered
Joined
·
900 Posts

·
Registered
Joined
·
87 Posts
Discussion Starter #5 (Edited)
I have changed the title to camera crop row guidance, stereo-vision is just one approach. I have gone through research on crop line recognition and a lot of it is done with purely visual camera's (algorithms based on features in image stills like color or shape). Reliability and accuracy varied. Someone even started an open source example of this: https://github.com/petern3/crop_row_detection

Recent papers most often use stereo- or binocular vision, the ones below integrate it with machine control using PID, pure pursuit, ... at quite accurate levels:
A Stereovision-based Crop Row Detection Method for Tractor-automated Guidance
Stereovision-Based Guidance Line Detection Method for Auto-guidance system on Furrow Irrigated Fields
Tractor path tracking control based on binocular vision

I am no expert but it seems that stereo-vision is the way to go. Interesting that for example IFM (and probably others?) offer relatively cheap hardware and software in a package ready to integrate in a range of applications, so for many to experiment with. Communication runs through CAN or ethernet I believe.
 

·
Registered
Joined
·
788 Posts
Very cool project - so specialized. OpenCV may also be an option depending on programming skill.

So is this the kind of system you are referring to? The control is insanely accurate.

Raven - SBGuidance implement steering - TWIN disc steering weeding - YouTube
yes raven twin disc - in austria for about 40.000 eur..

i have one farmer who want to try this with 2 systems of agopengps - one for steering the tractor and one for the machine..

need to ask for success...


btw i think ROS offers some lib for camera row detection
 

·
Registered
Joined
·
5,823 Posts
Not sure if you want to go full on diy, but there is tons of vision based line following with Pi and opencv using Python. it was a rabbit hole i was fortunate to escape from.


https://www.youtube.com/results?search_query=opencv+line+following


Also you wouldn't necessarily need a disc machine to set this up and play. You could just mount the camera on the front of a tractor, pick a crop line and go. (but i'm sure you already thought of this). Could it be as simple as a camera, a computer, a dribble of Python?
 

·
Registered
Joined
·
87 Posts
Discussion Starter #8
Also you wouldn't necessarily need a disc machine to set this up and play. You could just mount the camera on the front of a tractor, pick a crop line and go. (but i'm sure you already thought of this). Could it be as simple as a camera, a computer, a dribble of Python?
This is indeed the goal, adding only just as much hardware as needed while utilizing the autosteer already on board. The camera system would probably need a position precision of 1-2 cm like RTK. Frequency of position and heading info is the video frame rate, so up to 50 Hz. With almost continuous information on position, heading and steering angle I guess the autosteer could be very accurate too.
 

·
Registered
Joined
·
900 Posts
Are you wanting to do it purely via steering or are you better off doing it via a side-shifting toolbar? Ie gps for tractor and camera on toolbar for implement control?
Eg.
Crop Stalker
 

·
Registered
Joined
·
212 Posts
I remember watching a video similar to this one at a farm show about 10 years ago and was just mesmerized by it. If I'm understanding correctly; what you are suggesting is a lot simpler than this...

 

·
Registered
Joined
·
87 Posts
Discussion Starter #11
Are you wanting to do it purely via steering or are you better off doing it via a side-shifting toolbar? Ie gps for tractor and camera on toolbar for implement control?
Eg.
Crop Stalker
That's what we want to find out. It seems not a lot have tried and perfected direct steering of the tractor by camera for this application, maybe for a good reason, maybe not. In essence we want the hoe in the front and keep it simple.

If it really doesn't work (not accurate, reliable, ...) another idea of my colleague farmer is to have the hoe with camera and steering discs or sideshift, just like in existing examples but everything in the front. And then have a horizontally flexible parallellogram link between the hoe and tractor front. So the hoe steers itself and the tractor just pushes it (tractor should not even have RTK or autosteer, as long as you steer it in between crop lines). I should make a sketch.

Wilbert, that is crazy expensive stuff but it works flawlessly for bed-planted crops. The dream!
 

·
Registered
Joined
·
6 Posts
Here are a few thoughts to consider and a few reasons why the front cultivator mounted systems are not more commonly used:
- cultivator width is very limited because the wider you go the more the cultivator steers the tractor instead of vice versa
- the cultivator debt need to be adjusted perfect and only working rel. shallow otherwise again the cultivator becomes "un-steerable"
- if there is just a slight unevenness in the soil (planter track etc.) of rolling ground the cultivator pulls the tractor to that tougher side and just to stay straight you need to over-steer to the opposite side
- speed is limited (in a second pass you often want to go relatively fast (8 - 12km/h) to move some dirt into the row to cover up newly emerged weeds)

- because of all of this you need an oversized tractor in relation to the cultivator used
- narrow rows are not possible in a width of over probably 3 meter because you need some room to steer the cultivator and very narrow rows are not possible at all



We did use a front mounted 9m wide 12 row cultivator for awhile in organic corn & soys but needed to use a big 200+ hp tractor and still had lots of problems as the cultivator moves the tractor in an uncontrollable way and you always had to over-steer in the opposite direction of where the cultivator pulls the tractor. Just a slight adjustment of the cultivator had huge implications for the way it is steerable. The speed was limited to about 6 km/h, and that was often too slow for a second or third pass. So then we used the smaller tractor (95 hp). Another big disadvantage of the front mount cultivator system is the tracks the system leaves behind and the weeds germinate right away there again. Even in 75cm rows he tractor would stay not straight enough in the row to use a track eradicator at the back (because of the over-steering)


At the moment we cultivate 18 rows 50 cm row width and 24 rows 25 cm row width with a 90 hp front wheel assist tractor on 9" tires and it is a very smooth operation. Sometime I wish the tractor would be also steered by a camera mounted in the front (25 cm row width is too narrow for RTK gps based on my experience).





We finally went to a camera guided system at the rear 3pt hitch which works flawlessly in almost any condition as long as the camera or the crop feelers can detect the row.
 

·
Registered
Joined
·
87 Posts
Discussion Starter #14
Hello Marcus, thank you for your sharing your experience! We have discussed some of the points you make. I think our situation is slightly different from yours in these ways:
-width of 3m is fine
-depth is as shallow as possible, we try razor blading the top cm's of soil rather than cultivating. So force on the equipment is small.
-we use vertical plates and other small implements for pushing ground into the row, high speed is not necessary

The front implement causing sideways disturbance in steering is something we will probably have to overcome but we're hopeful it is doable in our situation. In my experience of manual driving so far, it is only noticeable when driving on a slope. The autosteer software could probably take this into account as Agopengps does now with the 'draft gain' parameter based on the roll.
 
1 - 14 of 14 Posts
Top