If you see a man photographing San Francisco’s garbage, it’s probably Emin Israfil.
Israfil has spent six years trying to use technology to make streets cleaner: Photographing sidewalk garbage, describing it, and attempting to create an up-to-date map of all the trash in San Francisco. And he has spent six years butting his head on the wall, unable to build a sustainable business model.
Enter artificial intelligence.
A new OpenAI model — specifically, API access to GPT-4 with Vision — made available to the public on Nov. 6 has been a game-changer in this niche market, Israfil says. It has made the description of trash a cinch — just point your camera, take a picture, and AI will describe every detail of the garbage in your photo, down to the brand of can or cigarette.
It may help Israfil obtain what he most covets: Contracts with government agencies, which he says can use his software to target high-volume trash areas, identify the most common types of refuse, and generally make their sidewalk garbage collection more efficient.
“We started quite naively,” he said. “We just see a problem, we want to solve it. Then we work backwards from the problem. But along the journey, we realized you actually should start, probably, with a customer, and then go.”
In a demo, Israfil dropped a McDonald’s paper bag on the ground, and then snapped a picture with his cell phone. Within seconds, his bright pink-themed app had categorized the trash by its material, and generated a description of the image.
“In the image, there’s a McDonald’s bag lying on the ground, next to a curb. The bag is the most prominent piece of litter in the scene. There are also a few leaves scattered around the area, but they are not significant enough to be considered litter. The McDonald’s bag is the only discernible piece of litter in the image,” read an emotionless female voice, accurate in all details.
“It helps identify the brand, the location, the composition,” said Israfil, who is now working to develop the demo into a full-fledged product that can be launched within the next two weeks. “It’s a huge game-changer. It just makes this data so much easier to gather. It just makes us as society, and governments, able to direct resources where they’re needed.”
Israfil, 36, goes by the moniker “garbage man,” and says his favorite phrase is “do something trashy.” He carries his pink trash picker around most of the time, and books himself two to three AI hackathons every week, working from 9 a.m. to 9 p.m to keep up with the ever-changing technology. The trash-themed projects he and his teammates build — “TrashGPT,” “RecycleGPT” or “PromptSF” — are often finalists in the competitions.
Through his startup, Rubbish — which he says is the only Bay Area company providing environmental dataset to clean up streets — Israfil had earlier tried to train his own AI image-recognition tool. That came after years of manually logging data points of trash on the streets.
The startup drew “boxes around stuff a million times” to teach machine images. They made some progress, but the model was not good enough. For example, it could only identify plastic bottles of a certain shape, or it could recognize cigarettes on a street, but not on a beach or in grass.
That was no longer the case when OpenAI Vision became available. The pre-trained model not only recognizes a wide range of trash, but also makes verification possible. “The only thing that was stopping us before is just the amount of labor involved in gathering this data, only small corridors could be mapped,” said Israfil. “But now, if you make it 10 times easier, it means you can cover 10 times the area, and help 10 times as many people with the same amount of resources.”
Another possible revenue stream: A new California state law that requires producers of garbage, like Coca-Cola, Marlboro, McDonald’s and other companies whose products commonly become refuse, to be held responsible and financially liable for cleanup. Government agencies first need to know how much trash belongs to which companies before charging them, and Israfil’s model may help them do so.
Trash first got on Israfil’s radar in 2017, when his friend’s dog choked on a chicken bone abandoned on the street in New York City. Consequently, the two friends founded Rubbish as a side project to organize cleanup events for tens of thousands of people across the country. At its peak, the social platform raised more than $100,000 in funding, allowing the startup to hire three employees.
They pivoted once the pandemic made trash-gathering social events impossible. They pinned their hope on selling advanced trash pickers that connect to phones, used Bluetooth connectors and came in an iconic pink color — their own version of the “Lyft mustache.” That plan also died as the pandemic made production costs too high; Israfil ended up making just over 100 trash pickers with a 3D printer in his room.
Since 2021, Israfil has helped build environmental datasets and partnered with various agencies, like a community benefit district and Sacramento’s river district. He has been trying to talk San Francisco Public Works into working with him on an annual trash map, something the city did in 2022.
He believes 311 trash data alone is not enough. “It’s mostly just a map of 311 usage,” he said. “It just shows the people who have smartphones, who are tech savvy, who speak English, who have the time. It’s not an accurate reflection of your city.”
After nearly giving up twice, Israfil is now the only full-timer at Rubbish, which he sometimes funds with savings from his early years working for a consulting company.
But regardless — probably handicapped by sunk costs — he’s still here, six years later.
“The question really relies, partially, on what’s new and exciting, and what you could do, how to use it to address older problems or intractable problems,” he said. “So maybe this AI technology unlocks blockers that have been in place for a long time around a lot of old problems.”
Thanks for covering this.
It’s inspired me to learn more about the real world side of AI and the technical underpinnings.
I really like this site.
The content/articles and, of course, the comments.
But also the web design. Simple and elegant.
If one does a View page source you’ll see some scary (for some) tech bugaboos:
googletagmanager
amazonaws
wordpress/Automattic – run by a $400,000,000 stereotypical “tech entrepreneur” guy
Think about that as you’re writing comments on your smart phone or thru a Chrome (Google!!) based browser.
AI can be a clever tool.
Intelligent – well – not quite yet.
Scientific and technological innovation does have its upside.
There will always be skeptics, naysayers, questioners – and that’s good.
But I’m really glad not to have to worry about polio – and that whole thing got off to a very rocky start.
It’s been going for long enough now that I feel I have to say, ML’s “AI beat” is an embarrassing thing to see continue. This stuff is all such a grandiose mix of “solutions in search of problems” and unsuccessful attempts to solve deeper social problems with technology – something the past decade+ has shown to have a very poor track record.
And the more harmless, light hearted experiments like this litter initiative would be much easier to stomach if this whole sub-industry weren’t currently in the midst of a full-court press PR + hype offensive, funded by some of the most socially corrosive forces at work in the world today – a previous article mentioned in passing, without any real acknowledgement, that a venture was funded by Peter Thiel, a literal blood-drinking billionaire fascist who wants to end global democracy and install tech monopolies as the de facto rulers of society! Then you have Sam Altman, a doomsday prepper whose non-AI hustle is a crypto scam to underpay people in developing nations for their retinal scans. The rogues gallery goes on and on. The political economy at work here is not hard to see, and ML usually has a more critical eye on this stuff.
These people only care about the problems of San Francisco insofar as they can control the “doom loop” narrative framing and thus sell “solutions” to them and accrue broader power over global policy. Uber et al have already taught us how that will go. They don’t care whether those solutions actually work or not (thus far, they sure haven’t – see ML’s positive coverage of Cruise, whose business is now in the process of imploding after a series of terrible accidents). And in a year or two when the current AI hype wave collapses, they’ll have moved on to something new, but these oddly credulous and optimistic articles will still be on this website.
One more time !
Approach Elon Musk to pay for a Million Dollar SF Trash Lottery.
Work an hour and get a lottery ticket with Morse Code one side, phonetic alphabet other side.
Manny’s could have given out 3,000 of them over last quarter.
City volunteer programs did 16,000 in 6 months ?
Yearly drawing July 4th on Stairs of City Hall.
Much better chance than winning regular State Lottery.
And, you can’t buy them.
You have to earn them.
How about just:
1. stop littering
2. make encampments clean up their own mess
3. get jailed people out into the fresh air and clean up as a condition for earlier release
4. immediately fine property owners/businesses whose sidewalks are not clean
5. stop dispensing coffee in unreusable cups
6. tax any food business that allows for take-out or for which one finds their identified trash on the street–only businesses that do over $250,000/yr. in net sales
7. don’t just respond to 311 postings; DPW: go up and down the streets that historically have the most dumpings and piles of detritus
8. when you catch a dumper, sentence them to two years-worth of cleanup and fine them, if they are a business.
How about just:
1. stop littering
2. make encampments clean up their own mess
3. get jailed people out into the fresh air and clean up as a condition for earlier release
4. immediately fine property owners/businesses whose sidewalks are not clean
5. stop dispensing coffee in unreusable cups
6. tax any food business that allows for take-out or for which one finds their identified trash on the street–only businesses that do over $250,000/yr. in net sales
7. don’t just respond to 311 postings; DPW: go up and down the streets that historically have the most dumpings and piles of detritus
8. when you catch a dumper, sentence them to 2 years-worth of cleanup and fine them, if they are a business.
So this AI app is able to identify garbage. That is very interesting, but this is not the sort of transformative AI that the city would benefit from the most.
How about developing an AI program that identifies our local government’s bloat and mismanagement, then identify performance metrics to generate a remediation plan with actionable rewards and consequences: expansion of activities that execute results and dismantling of activities that are ineffective.
This isn’t going to scale – it’s not only Singapore that isn’t getting trashed in the first place.