“There are few industries where an unnoticed mistake can cost as much as it can in healthcare,” says Matic Bernik, Data Scientist in our Innovation team, describing why he cares less about average accuracy scores and more about whether an AI system knows when to say “I can’t answer this reliably.”
Matic has been drawn to healthcare since childhood. His curiosity and excitement led him to a company that sold and maintained medical equipment, and then to Better, where he deals with data. He says he usually understands a domain by modelling it, and healthcare offers an unusually rich and consequential world to model. He believes that intelligence in healthcare AI shouldn’t reside entirely within the model, but in a system of narrow, verifiable capabilities working together, with clinicians firmly in control of the final call. Outside of work, Matic winds down with cooking, watching movies, and mountain biking, and he still finds time to argue with a chatbot about one problem or another. Get to know Matic in the interview below.
You are a data scientist in our innovation team, and you describe your work as “architecting AI systems for healthcare”. What does that actually mean in practice, and what does a typical day look like for you?
For me, architecting an AI system means deciding where the model is allowed to be clever, and where it is not. A language model can give answers that are plain wrong, in a very confident tone and with semi-believable reasoning, no matter the industry. But there are only a few industries where an unnoticed mistake can cost as much as it can in healthcare. So when I develop and evaluate these systems, I do not care only about an average accuracy score. I care whether the answer is grounded in authoritative clinical data or medical evidence, whether the ambiguity that came with it is still visible, and whether the system has a genuine way to stop and say, “I can’t answer this reliably,” instead of improvising.
As for a typical day, it depends on whether I am starting something new or continuing existing work. Early in the week, I lean towards research and towards indulging new and ambitious ideas: taking a problem and looking for a way to reshape it into some more standard form, then seeing who else has been fighting the same thing and what they came up with. I like to set an LLM on deep research and do my own exploration in parallel, and then compare the two. Where they overlap, I get some confidence that I have covered the main ground. Where they diverge is usually where the interesting part is and where I need to verify things more carefully.
Then implementation – sometimes of two or three competing ideas, because it is only while building that you find the sub-problems nobody anticipated. The best moment is when you find a way not to add a component but to remove one, and a single mechanism turns out to solve three of your problems at once.
And then testing, testing – reviewing the failures, reading the execution traces to see what the system actually did, tuning. I like to put tunable pieces into the individual stages of a pipeline as early as I can. A reranker, say: a small model whose only job is to push the most relevant results to the top, rather than asking one big model to be good at everything.
You work at the intersection of data, software engineering, and healthcare. What first pulled you towards this combination, and what has kept you interested in digital health?
I have been interested in natural sciences and biology for as long as I can remember, and my early encounters with healthcare institutions were marked more by curiosity and excitement than by anything else. As strange as this may sound, walking through the halls of University Medical Centre in Ljubljana had a cool vibe for me. Even when I was there as a patient myself, it was that curiosity that often carried me through the parts that could otherwise have been a bit scary.
From my late high-school years I also worked for a company that sold and maintained medical equipment. That gave me a very early, and very unromantic, look at what healthcare technology actually is: blood-gas analysers running on Windows 95, hospital systems disconnected from or heavily firewalled away from the internet, irregular upgrades, difficult database migrations – and staff who still needed their problem solved promptly.
I tend to understand a domain by trying to model it. A database schema may sound like a dry technical artefact, but it is really a compact description of a world: which things exist, which properties matter, and how they relate to each other. Healthcare gives you an unusually rich and consequential world to model. What has kept me here is a certain seriousness of purpose. The problems are hard, the constraints are real, and the consequences matter. It feels like work worth investing yourself in.
You have worked with clinical NLP and developed a Slovenian healthcare NLP pipeline that is now used nationally. What was the biggest challenge in teaching a machine to understand the language of healthcare, and what did that project teach you?
That was a great project and a very good learning experience. The pipeline was eventually deployed within Slovenia’s Central Registry of Patient Data, but the difficult part came much earlier: defining the task, and creating data that represented it.
The first challenge was data. With any data-driven system, the development data needs to resemble what the system will actually meet in production – similar vocabulary, structure and messy details. Clinical text is difficult to access, for good reasons. And Slovenian is spoken by roughly two and a half million people, so models, corpora and other resources were already hard enough to come by at the time; for Slovenian clinical language there were essentially none. So we had to build and annotate much of the dataset ourselves, among other things by translating publicly available clinical datasets from other languages, and bringing in a group of clinical experts to check that what came out was actually correct.
Then came the harder question: what do we even mean by “understanding” a clinical document? Which things should be extracted – medications, diseases, signs, symptoms, anatomy, procedures – and how detailed should those categories be? Take fever. In one sentence it may be something the patient reports, in another a measured sign, and in a third part of a diagnosis. If the categories are too detailed, a limited dataset may not contain enough examples for the model to learn the difference. If they are too broad, the system is easier to build but less useful. And if the annotation rules are vague, not even the experts will agree consistently – so the model has no stable target to learn from.
Generative AI makes it much easier today to produce candidate synthetic examples, but that does not remove the problem. Synthetic text can miss the messiness and the distribution of real clinical notes, so it still needs careful expert and statistical validation.
The main lesson was that a single end-to-end model is often not the best production solution, and almost certainly not in the early phase. We got far more predictability by combining statistical models, deterministic rules, clinical terminologies, and local expert knowledge. And the system has to leave room for hospitals and users to express their own conventions. That is not something you bolt on at the end; it is part of the architecture.
You said that in healthcare, how you architect an AI system can matter more than how clever the model is. Why is that, and what separates a fascinating AI demo from something clinicians can genuinely trust?
Models are very good at charming you. You ask a few questions, the answers sound knowledgeable, and it is easy to mistake fluency for dependable understanding. I include myself in that – it is a pleasant feeling, but sadly not one you should build critical systems on.
A demo only needs to work on the examples you already thought of. A clinical system has to survive the ones you did not. A record where the allergy field is simply empty – which is not at all the same thing as “no allergies”. A code that was valid three years ago and has quietly been retired since. An abbreviation that only one department in one hospital uses. Or two documents that disagree with each other about the same medication.
That is why I try to give the model a narrow role. It is genuinely useful at the boundary with people, understanding a request in natural language, explaining a result, or producing structured output such as a query that can be validated before it runs. But factual claims should come from approved sources, and the evidence and the steps behind an answer should be inspectable.
A simple example is clinical coding. If someone asks for a medication code, I do not want the model to remember or invent it. I want it to call the approved terminology service and return what is actually there. If the service finds no valid match, that should be visible. For this kind of lookup, no answer is better than a plausible wrong one.
The system can also check its own work, and the checks that matter are the ones made against something outside the model, even when a model is the one doing the checking. Does this code actually exist in the terminology? Does this query parse, and does it only reference fields that actually exist? Does the claim in the summary actually appear in the document it cites? Those checks can fail, and because they can fail, passing them means something.
And a failed check is not just something to log afterwards. It is a decision point, while the answer is still being assembled. Sometimes the system can repair the problem itself – re-plan the query, try the lookup a different way. Sometimes the right move is to go back to the person and ask which of two things they meant. Sometimes it is to return the answer with the discrepancy shown alongside it. And sometimes it is to decline. What matters is that the problem surfaces while something can still be done about it, instead of being discovered later by whoever trusted the answer.
None of that makes the system foolproof. Checks catch the kinds of mistake you thought to check for, and an answer can still be wrong in a way that passes every one of them. But it narrows the space considerably, and what is left is at least concrete enough to inspect and challenge.
And when several pieces of evidence compete, the clinician should see them and make the decision. I see the useful future as guided cooperation, not a handover.
AI is moving very quickly into healthcare, from clinical documentation and decision support to more autonomous “agentic” systems. Where do you see the biggest opportunities, and where should we be particularly cautious?
The biggest opportunity I see is in the work that surrounds care without being the care itself: documentation, coding, finding the one relevant fact in a long record, and bringing together information that is split across several systems.
“Agentic” sounds more dramatic than it has to be. An agent can simply be software that understands a request and chooses among a limited set of operations it is allowed to use. One component can retrieve data from an EHR system, another can look up clinical codes in a terminology service, another can validate the request, and a coordinator can bring those pieces together behind one ordinary question. The useful part is not one model that supposedly knows everything. It is several narrow capabilities cooperating.
I become much more cautious when a system can do things, and not only say them. A draft can be reviewed. Updating a clinical record, placing an order or triggering a workflow has a different failure cost. As autonomy grows, each permission should be scoped more tightly, important actions should be validated and confirmed, and the system should leave an audit trail that shows what happened. Some mistakes can be corrected; some cannot be taken back once a person has acted on them.
There is a lot you can do to make writing safer. You do not let the model compose freely into a record, you constrain the write to a template, so it is filling defined fields rather than writing the entry itself, and you validate what is about to be written before it is committed. That catches a great deal.
But it is worth being honest about what it does not catch: a write can be perfectly well-formed, pass every technical check, and still be the wrong write. And once it is part of the structured record, it carries the same authority as everything around it. Nothing marks it out as the one that was wrong.
So “agentic AI” is not one level of autonomy. It is a spectrum, and we should be precise about which part of the responsibility we are actually handing over.
You said you are driven by the fact that, if you look at data the right way, it can reveal things that haven’t happened yet. How does that idea influence the way you work?
Ah yes, there are really two things that attracted me to data science. One is the amount of domain understanding you need to acquire on the job so that you can model the data properly at all. The other is predictive analytics, if you have enough context and a deep enough understanding of what happened in the past, you get a glimpse of what is most likely to happen next. I still think that is, for lack of a better word, cool.
Although there is some irony in where that has actually led me. Working out what is going on with a patient and what is likely to happen next is the doctor’s own job, that is what they trained for. What genuinely robs them of time is everything around it: the administrative work, and the death by a thousand clicks it takes to get one clinical question answered by the system.
So the prediction business we are mostly in, day to day, is predicting the next token – roughly, the next word. Which is what a language model is trained to do, underneath all of it. I came to data science to say something about the future, and I spend most of my days on clinical language.
What is more interesting to you: predicting what will happen next, or understanding why something happened in the first place?
I see the two as very closely connected. My instinct is that if you enjoy trying to predict what will happen next, you will probably also want to understand what actually produced the outcome in the first place. It is the same analytical curiosity, just looking in two directions.
Though strictly they are not the same thing. A model can predict well from a pattern without anyone knowing why the pattern holds, and a feature can be highly predictive while telling you more about how a particular hospital works than about the patient. That holds up nicely until the setting changes, or until people start acting on the prediction and change the very thing that produced the data.
So if I had to choose, understanding why is the more interesting one. Especially in healthcare, because predictions are usually there to support decisions. A risk score can tell you that somebody needs attention. It does not, by itself, tell you which intervention will help.
Prediction is still a good reality check, though. If what you think you understand does not survive contact with new data, something is probably missing from the picture, although sometimes the missing thing is simply that the other hospital codes things differently.
AI and data science are changing so rapidly. What are you currently learning or experimenting with that you think could significantly influence healthcare in the next few years?
Right now I am interested in a fairly practical question: how much of the intelligence really has to live inside the model, and how much can live in the system around it? A lot of our experimenting is about giving a model a clearly defined set of things it is allowed to do, and then seeing how small and specialised that model can be. But the part I find genuinely exciting is what that could do to the software itself.
Today the clinician adapts to the application. You learn where the screens are, which tab holds which piece of information, and how many clicks it takes to get from a question to an answer. The workflow was fixed in advance by whoever built it, and everybody bends around it.
If instead you have a set of agents that each know one data source or one task well, they can compose a workflow on the spot for the question that was actually asked. The application adapts to the clinician, rather than the clinician learning to adapt to the application. Dynamic would not mean arbitrary, though – the building blocks stay fixed and approved, and so does what each of them can do. What changes is which ones get used, and in what order.
Would I bet on that arriving within a few years? Honestly, I am not sure, and the reasons have little to do with the models. Composing a workflow on the fly assumes that the underlying data is reachable and can be interpreted consistently across systems, and in many places, neither is true yet. Healthcare is slow to change, and given everything I have said about verification, I am not sure I should complain about that.
What takes your mind away from the world of data and technology? What do you do in your free time?
Since childhood, I have liked watching movies, and I still do whenever I have the time – although, more often than not, I spend almost as long deciding what to watch as I do watching it.
I like catching up with friends over coffee, particularly when the conversation moves beyond the usual updates and takes on a life of its own.
I like travelling – the kind that makes you feel free. I also like the flowy sensation of cycling and mountain biking: the climb, and then the rewarding descent on my Orbea Occam. Mostly, though, it is simply about moving through nature, among the trees or alongside a river.
Cooking is what helps me wind down after work. It probably uses a different part of my brain.
I am not very good at keeping technology out of my free time, mind you. A lot of that time still goes into reading, searching, and arguing with a chatbot about some problem or other.
What does Better mean to you?
Better, to me, is a particular kind of crowd: drawn to healthcare because the problems there are hard and worth solving, bothered when something is not done properly, and set on actually finishing things. That combination is rarer than it sounds. There are plenty of good ideas in technology. Getting one to survive real data, real users, and the systems that are already in place is a different job, and most of it happens long after the idea stopped being exciting.
