For years, Mark Hager worked as an at-sea fishery observer, going out on New England fishing boats for days or weeks and keeping detailed records of every fish caught or thrown back. The work could be grueling and dangerous; on one trip, a scallop boat turned sideways in 20-foot seas, and Hager and the crew put on survival suits in case they had to jump overboard. But the counting was essential to protecting the ecosystem, helping federal regulators track endangered species, and ensuring that no species was overfished.
In the early 2000s, the fishing industry began installing video cameras in strategic spots on boats, so that humans could track the data from ashore. In 2019, Hager and the Gulf of Maine Research Institute launched a company, New England Marine Monitoring, based in Portland, Maine, to provide technology support for vessels using electronic monitoring. The process, while less harrowing, is still slow and tedious: His team has to watch hours of video footage, look for each moment when a fish is discarded, then make a note of the species and the time it was tossed. In ten hours of video, there might be 45 minutes between each case of a discarded fish.
When Hager consulted with scientists at the Roux Institute at Northeastern University, they hatched an idea: Perhaps artificial intelligence could do it better.
Now, working with electrical and computer engineering professors Octavia Camps and Mario Sznaier, Hager and his team are using their notes as training data for an AI algorithm — programming the AI to scan the video footage and indicate points of interest along the timeline for a human to look through.
“Instead of ten hours of video, we’ll be able to look at about 100 pictures, which we can do in about 20 minutes,” Hager says.
The result could yield huge amounts of data that would be valuable to scientists and fishermen alike.
The result could save time and money, but Hager has a bigger goal. He wants to prove that AI algorithms can be used to count every fish that’s caught, in addition to every fish that’s discarded. Discarded fish make up about 10 percent of the total catch, while kept fish make up about 90 percent. To be effective, the algorithm will need to be able to identify the total volume of a fish haul, count containers of fish, and potentially even count and measure individual fish.
Using video monitoring to count a small fraction of the total catch is one thing. Using it to count the entire haul on a vessel is a huge feat — and one that has never been accomplished before.
“But with a human-machine connection, we think it can be done,” Hager says. And the result could yield huge amounts of data that would be valuable to scientists and fishermen alike. For example, as the climate changes, researchers could use the data to understand how the distribution of fish species in the ocean has changed over time and predict where fish will shift in the future, Hager says.
New England Marine Monitoring and the Roux Institute, which is also based in Portland, recently received a $225,000 grant from the National Fish and Wildlife Foundation to develop the algorithms. The goal is to help not just Hager’s company, but the entire fishing economy in Maine, says Michael Pollastri, a professor of chemistry and chemical biology and the academic lead for the Roux Institute — which provides research and technology support to more than 40 Maine businesses.
“We’re partnering with folks to make sure we are tailoring the education and research opportunities,” Pollastri says. “Everything we’re building here is based on a stated need by partners, or the need of the sectors that we’re looking to grow here in Maine.”
Published on
