comments:
I don’t think you’re crazy, but I’d separate Scrum from the implementation you’re describing. Scrum can work outside pure software, including data science and data engineering, but only if there is a real product, a real Product Owner, a clear outcome, and some usable increment or learning every sprint. If people are farmed out across a matrix, no one owns priority, there are no Product Owners in the reviews, and a deputy director is using the forum to judge progress, that sounds less like Scrum and more like old command-and-control management with Scrum vocabulary pasted on top. The “usable deliverable” issue is fixable, but it has to be interpreted correctly. In data work, an increment might be a reproducible dataset slice, a working pipeline step, validated access to a data source, a model benchmark, experiment results, a data quality check, a dashboard prototype, or even a decision that a hypothesis is not worth pursuing. But “I called a guy to ask for database access” is not really a deliverable. That is a blocker or dependency. It should be visible, but it should not be treated like value delivered. Over the past several years, I've acquired a pretty good library of productivity books. Some of the one's that I would suggest reading are these... This is where I think Joshua Seiden’s Outcomes Over Output is useful. The question should not be “what did you produce this sprint?” in a narrow factory sense. It should be “what changed because of the work?” Did a decision get made? Did uncertainty go down? Did a stakeholder gain the ability to act? Did users or analysts behave differently because of what the team learned or delivered? David J. Anderson’s Kanban: Successful Evolutionary Change for Your Technology Business is probably even more relevant to your situation than strict Scrum. Data work has a lot of waiting, access requests, approvals, upstream dependencies, blocked work, and invisible queues. Kanban gives you a way to visualize that reality, limit WIP, manage blockers, and improve flow without pretending every two weeks neatly produces a finished product. Donald Reinertsen’s The Principles of Product Development Flow also speaks directly to this. Data science is high-variability knowledge work. Trying to manage it like predictable factory work creates nonsense. The better lens is flow, queues, batch size, feedback speed, WIP, and cost of delay. If everyone is overloaded and work is stuck waiting on access, reviews, approvals, or SMEs, more check-ins will not fix that. Ryan Ripley and Todd Miller’s Fixing Your Scrum is useful because it calls out a lot of what you’re describing: daily Scrum turning into a status meeting, trust being missing, Product Owner problems, sprint reviews becoming judgment sessions, and teams doing “mechanical Scrum” without the values. That book would probably give you better language for saying “we are not doing Scrum well” without sounding like you’re just resisting change. Jeff Sutherland’s Scrum: The Art of Doing Twice the Work in Half the Time is useful as the basic reminder of what Scrum is supposed to be: inspect and adapt around real progress, get fast feedback, expose impediments, and let small teams self-organize. It is not supposed to be a recurring adult babysitting ritual. If I were in your shoes, I would not argue “Scrum is bad for data science.” I’d argue that your implementation needs clearer product ownership, better definitions of valuable increments for data work, explicit blocker and dependency handling, less status theater, and reviews that are about learning and decisions instead of judging people. Your frustration sounds legitimate, but the better target is the operating model, not just the word Scrum.