Priscila Oliveira on Sentry's TypeScript Migration
The migration was driven by the need to address front-end bugs and the desire for an improved developer experience.
Priscila outlines in detail how t
0:19 I'm super excited to hear about Sentry. It's also one of very few open-source codebases out there that are actually driving an actual business. It's super valuable as a learning resource and I can't wait to hear all about that.
0:35 Priscila, why don't you introduce yourself, say what your history is with developments and how you got to Sentry.
Priscila Oliveira: 0:41 Sure. First of all, thanks for having me. I'm very happy in participating in this podcast. My name is Priscila. I'm from Brazil. I graduated. I have a bachelor in computer science.
0:58 I always liked working with computer. I still remember when I had my first big computer at home and I was playing game the whole day. I really enjoy working with computers, algorithms, and everything coding.
1:17 I had a big dream. It was about moving to Europe. I always dreamed of living here. I also like to learn all the languages, human languages. I learned German, Italian.
1:36 After some years living in Austria, I moved to Czech Republic where I had my first contact with React, and also Flow type. After, I moved back to Austria, where I had my first contact with TypeScript.
1:51 After some years back in Austria, I got my job at Sentry. I am working at Sentry as a software engineer for a little over three years now.
2:06 My current team in Sentry is called Telemetry Experience Team. We are mainly responsible for data insights in our product, such as dynamic sampling.
2:18 I do some backend work sometime in Python. Our backend is in Python, but my focus is on frontend. If the team needs anything, a new UI for a feature, probably it's me who is going to implement this UI, for example.
Matt: 2:40 Nice. Sentry itself, I think a lot of developers will have heard of it. It's been advertised on Syntax.fm, and things. It's an error monitoring tool. It's used in lots and lots of different places.
2:50 I've put it into production myself. I've used Sentry. I'm sure lots of people have. Is that an accurate characterization and you're working on the frontend for it?
Priscila: 2:59 Yes. Sentry is open source. We have a great product. We have the error monitoring product. This is our main product. We also have Performance. You can use Performance to check, for example, if your website is slow and the problems, how you can fix it.
3:21 Recently, we also launched Replay. You can now see a video of how a bug was triggered, for instance. We also have Profiling now. You can use Profiling and see what's wrong.
Priscila: 3:44 Yes, exactly. I love Sentry. I use the product myself every day, all of us at Sentry. We use TypeScript, for example, for catch bugs during compiling time.
4:02 It's really helpful. Sometimes -- I'm not talking about TypeScript, but some another bugs -- we have tests, we have a bunch of things to try, trying to prevent shipping bugs to production. When this happens, then we use our product to monitor. Then we can see what's wrong, what kind of regression we...yeah.
Matt: 4:26 Just like TypeScript is built in TypeScript, Sentry is used to monitor Sentry. That's pretty funny, I like that.
Priscila: 4:32 Exactly, we do that.
Matt: 4:33 Nice. That's so cool. You're on the Sentry team.
Priscila: 4:38 Yes.
Matt: 4:38 Let's talk a bit more about Sentry's kind of codebase and because it's so cool that so much of it is open source. It can't all be open source. Is there some stuff that's closed source or...
Priscila: 4:51 Yes.
Matt: 4:52 Or, maybe you can't say.
Priscila: 4:53 No, I can say. I believe so. We are really very free at Sentry to say a lot of stuff. I guess that 90 percent of our code is open source. A very small part of our code is not open source. I almost don't work on this private repository. It's mostly for billing, for, what's the name? Sorry.
Matt: 5:28 The more secure stuff, I imagine.
Priscila: 5:31 Yes.
Matt: 5:32 authentication...
Priscila: 5:32 Plans, for example, the plans you have, your team have or business plan, your company, it's more for this things. Most of the work, I think you can go today to the Sentry repository and you can see everything I have done today, all the PRs. Everything is public.
Matt: 5:53 That's amazing. Before we talk about the TypeScript stuff, what led to that decision? Why is so much of the code open source?
Priscila: 6:04 This is a good question. I myself had to learn it. I had the same question when I joined Sentry. The story I know is that David Cramer, one of our founders, when he started the project, the Sentry version one, it was based on another open-source project. He wanted also that everyone uses Sentry. This was his idea, "Open source, let's keep it."
6:39 I know that after he met Chris Jennings, another co-founder, they started working on a startup together. This startup was already using the Sentry version one. They liked it a lot. They also noticed a lot of things that could be improved. They've been increasing these day slowly, they were improving the product, the Sentry.
7:13 Only after two years, they really had a good product because they put a lot of thought into it. Then it was Sentry version two.
Matt: 7:26 It was open source from the start and...
Priscila: 7:27 From the start, from the first day.
Matt: 7:31 Then in the Sentry code base itself, what is actually in there? I talk a lot on Total TypeScript about the difference between application code and the level of code that you write, and especially the level of TypeScript that you write in your apps, let's say, which is usually fairly simple, usually consuming libraries.
7:51 Then you might also get library code, which is code written in reusable libraries and packages and things, which is a lot more complicated. What code is in the Sentry's code base? Is it like a monorepo or is it just one big app, or how does it work?
Priscila: 8:09 We have a mixture of a lot of things. We have some separate repositories, for example, our platform icons. If you're on board, you are going to see a lot of icons you can choose from so you can start your project. These icons, they are in a separate repository.
8:33 Most of the things included our component library is in our monorepo, is in Sentry. Then we have our Python code. We have our front-end code. We have our component library. Most of the things are there.
Matt: 9:14 Then you decided to move it over to TypeScript at some point. What led to that decision? I think that was slightly maybe before your time there, right?
Priscila: 9:23 Yes.
Matt: 9:23 Do you know the history of that decision?
Priscila: 9:26 When I joined Sentry at the beginning of 2020, the migration was already going on. Of course, I know why they decided to go with TypeScript. I know that we were shipping more front-end bugs than expected. This was not nice. That's one reason why we decided to go with TypeScript.
9:56 Also, because TypeScript allowed us to do this gradual migration. We didn't want to just run, I don't know, some script and convert everything to TypeScript at once, adding any types everywhere. We'd like to do a great job and take our time, and see what's the correct type for a component or...
10:28 We wanted to do a great job, and of course, sometimes we had to use any, but we did it slowly. That's one of the reasons we decided also to go with TypeScript. There are more reasons, like the community. It's really active. They release all the time something good and they are fixing bugs all the time, so it's great. The documentation is good.
11:05 Let me think what more. There were so many good things and we decided it's going to help us. One thing also was about, if you use IntelliSense in a code editor, you're going to get hints of types. For example, this prop is boolean or string. You can get these kind of hints and this helps a lot, a lot. I love that. This auto-completion, these things. It improved our developer experience, I would say.
Matt: 11:53 Then when you're deciding to do that, there's obviously a lot of good things like you described just there, but there's also bad things, which is, this is...How big was Sentry's code base like at the time when you decided to do this? It's pretty enormous now, right?
12:11 You knew when you started it was going to take a long time to get through it. How did you justify that when you could have been working on features, for instance?
Priscila: 12:22 It was, according to our -- I had to check here, our blog post -- it was 1,100 files. We had to convert a lot. The thing is because we decided to go with, say, gradual migration, we could do it slowly when we had time. Our focus is always product goals. We work with sprints. Different teams, they have their sprints, and we have our goals for the quarter.
12:53 TypeScript was a side project, but everybody was super excited about TypeScript. After when people saw the benefits that it brings, people got even more excited. [laughs] Also because we agreed that we wouldn't use other cool libraries out there or React Hooks, for example, until we completed the migration. It was just an agreement.
13:25 Some people like me, for example, I came from a background where I was using TypeScript, I was using React Testing Library, and I was using React Hooks. I really wanted to use React Hooks at Sentry. I like Hooks, so I was helping the migration and pushing forward together with my colleagues so we could, as soon as possible, use Hooks.
Matt: 13:50 That's interesting. I don't think that made it into the article, actually. I like the idea that you're dangling a carrot of React Hooks so that you can get to the end of your TypeScript migration. That's kind of funny. How did that decision come to be then? Because you could have adopted React Hooks. You could have done that halfway through.
Priscila: 14:11 Yes. I wasn't yet part of Sentry when this decision was made, but I believe that's because...It's not nice when you start something and then you start another thing at the same time. Then it's a mess.
Matt: 14:51 How many of you, in terms of the development team, were involved in this? How big was the team that did this?
Priscila: 15:02 A lot of people, but I believe it was a little over than 12 people.
Matt: 15:08 OK, 12. Reasonable to communicate altogether and things. In terms of the team dynamics then, were there a couple of people on the team that were really, really good at TypeScript that were teaching other people or was there a base level that was very, very good? How did that work?
Priscila: 15:27 Yes, in the team, we had some people already with this TypeScript background. These people, they were helping a lot other folks, especially during code review. I think one key for a successful migration is code reviews. It's a really important thing for this education process in a migration.
15:56 These folks, every day, they would set aside some time to do code reviews and help other folks. Once these new folks would be educated and learned a bunch of things, they would also help other folks. This was amazing. The education part of this migration, I love that. It was amazing.
Matt: 16:19 A feel-good experience. Nice.
Priscila: 16:22 It was nice.
Matt: 16:24 That's so cool.
Matt: 16:26 Sorry.
Priscila: 16:27 Sorry. Some folks, some people, they knew more about something else, and some other folks about another thing. When there was some gotchas, people would share it in a channel with other colleagues or during our front-end TS meeting. We were communicating a lot and sharing a lot of information with each other. This was great. I loved that. It was really great.
Matt: 16:58 That's awesome. That's an amazing experience. Code review is really important at Sentry. You guys do continuous integration and you do a lot of testing too to make sure that nothing breaks.
17:14 Could you walk me through how things work at Sentry? We're taking a detour away from the migration a second because I think that's important to understand that you guys like "push to main" a lot and you deploy a lot. How does that work at Sentry?
Priscila: 17:31 Every day, we open a lot of pull requests. I had a number, I cannot remember right now, but it's a lot of pull requests on a daily basis. As soon as we get one approval, usually from other folks also, for example, if I'm creating a front-end PR, then I wait for a review from another front-end, for example.
17:57 As soon as I have one approval and all my tests pass, then I can merge my PR, and this PR is directly deployed to production. This is how it works.
18:09 Mainly, of the things we work on, features, we hide them. [laughs] We hide them behind feature flags. Only us can see these features and we can test and do our things. Also, many times we are just directly pushing to production. Then we use our Sentry products to monitor our changes. That's how it works.
18:38 We usually monitor releases. We see the last release. We start commit, and then you can see if some show happened, some new show. Sentry is great. You can see the exact trace. You can see if it was you.
Matt: 18:58 Yeah, that's pretty cool. I hadn't thought that the actual product itself helps you be more confident in your...That's very nice. Nice dog feeding.
19:06 Oh, I had a question. What was my question? How does TypeScript work in that equation specifically? Do you lint the entire code based on every commit? Do you run it on CI? How does it work?
Priscila: 19:20 Sorry, can you repeat your question?
Matt: 19:22 Yep. How does TypeScript work in your CI? Do you lint on every single commit? How does that fit into your continuous integration?
Priscila: 19:43 We use GitHub actions to run our tests in CI. Usually, we have this package.json a script, right? To run your TypeScript, to run your lint, everything, like the commands. The CI will run them too, as well as you open your request and always when you commit something.
Matt: 20:09 Nice, yeah. That's the way that most people do it as well. There's also some discussion is like, should you run TypeScript like on your dev server as well?
20:19 Something I've asked to a couple of people is, would you like it if...? Let's say you're working on your front end app. If you've got a type error, there is some setups where it like freezes the screen and you can't look at your front end code until you fix the type error.
Priscila: 20:35 Oh, I don't like this. We had some cases. I run into some problems like this before, but now it doesn't freezes for me anymore. I think someone changed some configuration.
20:51 I do see the errors in my terminal, and I'm happy for them because then I fix all of them before pushing. Also, when I'm pushing my commit, we run every check locally.
Matt: 21:08 Oh, interesting. You do run your CI stuff locally before you can push?
Priscila: 21:15 Yeah, it's like it will look for TypeScript errors. It will look for link issues, things like that, but if it's a pretty issues, just if the code is in the format the team expects, then this CI is going to fix it for me.
Matt: 21:36 That's nice. It runs prettier for you.
Priscila: 21:38 Yes.
Matt: 21:39 Oh, that's very nice. Let's go back to the migration thing then. You're in the migration. There's 12 of you working on it. You're all learning a lot and got this lovely collaborative atmosphere.
21:54 How do you track your progress during the migration? What are you doing? What tools are you using to work out how far you're through?
Matt: 22:03 In the very beginning...Yeah, not in the very beginning, because I arrived later to the party. We had a file. It was a list. I think we have also a screenshot in that blog post I wrote, but it was a list we had in Notion.
22:30 With all the files, which still needed to be converted and who was going to convert that file, but I heard that this worked well in the beginning, but after it didn't. People wouldn't check this list anymore.
22:50 Me, for example, I didn't remember that list even existed, and I would just convert files. What really worked for us, it was like...Yeah, I remember that people were converting all the easiest files. Many folks, they would run a script in a terminal to see how many lines a certain file had. If they had very few lines, then this is the file that they would convert, you know?
Matt: 23:22 [laughs] Yeah.
Priscila: 23:24 I remember that. I remember cases when people would convert the same file. Unfortunately, this happened, but we were cool with that. It was not a problem because it was not a lot of code, right?
Matt: 23:42 Mm-hmm.
Priscila: 23:46 The large files, people actually they were avoiding them because they were complex or because they had a very complex types.
23:56 We converted them in the end of the migration, but it was just like that. Usually, you convert. Then you create a request and other folks review and it was like that.
24:11 Myself, one thing that worked for me was...First of all, because I'm in a different time zone, and back then, we had only two front-end engineers in Vienna, so me and my colleague. We would talk and synchronize between us.
24:30 Every day, we would go to the notifications and GitHub notifications and see the pull requests of files that people already converted in other time zones. For example, in San Francisco, they are nine hours behind us. It worked pretty well for me because of these two.
24:51 In the end, I don't know, in the middle of the migration, a team responsible for many tooling we have at Sentry, they created a bot for our channel.
25:09 Then people could just run a script, I don't remember, TS percentage or something like that. Then they could see the percentage we were at and the remaining files that needed to be converted.
Matt: 25:28 I imagine that's pretty cool. At the start when you see it going up, but when you're at the 52s or something, going to 53 and thinking, "Oh, God, there's so much to go on 110,000 line code." That's crazy, too.
Priscila: 25:40 Yes.
Matt: 25:42 How long did it take like over the course of the whole thing?
Priscila: 25:46 Oh, I think it took a little bit. It started in 2021, and we completed in 2020. Wait, wait.
Matt: 26:08 Minus a year.
Priscila: 26:13 Yeah, I'm terrible with dates. No, wait.
Matt: 26:17 It took you minus one year to do it. That's incredible.
Priscila: 26:21 Yeah, it took a long time.
Matt: 26:28 I noticed, too. I was looking around the code base. I saw there are still some files with .jsx endings, I think mostly tests actually. I imagine it's still a small amount of work in progress.
26:41 What I was interested in from the article was...In the article, I noticed that there's a really cool section where you say, when you're doing a migration, you should work out from the core, which sounds like a Pilates thing, first of all.
26:56 Then you've also said like, when you should go from the core outwards, in other words, you should look for the files with the most dependencies, the files that are most crucial to your project and start converting those first and then like work out from there. What led you to take that approach? Because I think that's a good approach.
Priscila: 27:21 Yeah, it sounds a good approach, but unfortunately, I was not part of...I don't know why they did this.
27:30 When I joined at Sentry, people had converted those files with this approach. I just started converting [inaudible] . I didn't use this approach, but I think it's good. It's a smart one.
Matt: 27:48 Why do you think it's good? What's good about it?
Priscila: 27:52 Converting the files that are used in most of the places. I can think now, for instance, about components like a button. A button is used in so many places.
Matt: 28:14 I guess if you don't convert the button first, whenever you use the button, you're going to have to...You've probably got worse types there. It makes sense to get the core stuff right so that then it filters out into the rest of it.
Priscila: 28:31 Yes, exactly. If you're converting, I don't know, a form, and this form, maybe you converted the whole form in TypeScript but this button is not in TypeScript, then you're not going to have a completion, for example. You are not going to know the types of this button, so you have to go to the file to know.
28:55 Maybe things are wrong there and you are going to run into some troubles because you think that the type is string but actually the type of something, it's a boolean. I think it's a really good idea to convert first the core components of your application, and after then you convert the rest.
Matt: 29:20 It's complex, though. Because that's often the root stuff, the core stuff, is where the most complex TypeScript is going to be. That's often the place, because you're using it in lots of places, it's going to need to be generic.
29:31 There's probably going to be some generics in there, there's probably going to be some more advanced stuff. That's probably the place that you're more likely to use anys that you're more scared of.
29:42 I talked to Shondai Person about this as well, who works at Netflix and she's done some migrations there, and she said it's good to have some people working on the leaves and then some people working on the roots. It depends on how confident you are in TypeScript.
29:59 If you feel that you can only do little bits, then you should start on the leaves and work inwards, because then you build up an understanding. If you feel pretty confident, then you can go from the root outwards. What do you think about that?
Priscila: 30:13 This is interesting. I like more the idea of, first, start from the very beginning and do a simple job. Convert the button for the component you know that's using this button. Then when you go to another view, another component, like a group of components, a page, for example, using this button, and you notice that, "Oh, actually the button also would need this type, this sort of thing."
30:49 Then you are using generic, suppose that you are using generic in your button, then you correct the type there in the button. I think it's a progress. You always have to correct your types as you go. This is super normal, but I like this idea of first converting the core components and after going to the views. I think it helps, in my opinion.
Matt: 31:15 No, I agree too. I think it's true too that I like that you guys chose a solution where you didn't just convert all the files to TypeScript and you didn't put a load of anys in there. Because I think there's a CLI called ts-migrate that does that for you, that just go brrp, and then suddenly everything's there. How come you chose this more incremental approach?
Priscila: 31:40 Why did we choose that?
Matt: 31:43 Yeah.
Priscila: 31:43 First because, as I mentioned before, we didn't want to rush. We wanted to introduce the correct types to our components. We also had to focus on product goals. We couldn't just migrate this large code base from one day to another. We had to do it incrementally.
32:12 We tried to do our best and check for the correct type. When this was a little bit complicated and it was slowing us down, made some generic or...
32:24 Yeah, we had some complex types and then we had to choose the any, but then we always added this to-do notation. The idea was to, after the migration, go there and correct those types because some of them were really tricky.
Matt: 32:43 What kind of tricky ones did you run into there? What do you remember?
Priscila: 32:47 I remember this forwardRef thing. We couldn't use the generic types there if I remember well, so it was a pain in the ass. Sometimes I had to use any.
33:07 We had to do this compromise to use any. We tried a couple of solutions, but back then, we couldn't find anything so we just used any. It was a little bit frustrating because we wanted to have the correct type for the things we were doing.
33:24 Recently, I also heard on Twitter, someone sent me a link from an article talking about an alternative solution for this forwardRef. I still have to check if this is going to work for us. Probably yes. Now we have a couple of forwardRefs to fix, to introduce better types in our code base.
Matt: 33:53 I think I know the blog you mean. It's by Stefan Baumgartner. I'll link it below. It's a really genius solution to that particular forwardRef problem. It's so good.
Priscila: 34:05 Yes, exactly. I still didn't have time, but I plan to do that. I think people are going to be happy about that because I think here we still use the same approach until nowadays.
Matt: 34:16 You're done with the migration but you've still got some of these holes then in the application where you can still look to improve the types. What does that process look like? How are you tackling that? Because I guess that's not like an official migration, right?
Priscila: 34:34 No.
Matt: 34:35 This is the lay of the land now.
Priscila: 34:37 Yes. For example, we run into this issue with the default props because we used to have -- we still have -- a lot of higher-order components, and the default props don't work well with them in a class. It's something like a side project. If someone is touching that file for some reason, then refactoring or introducing some new feature, then we improve the file.
35:08 It usually works like that because it's so large and so complex, and we cannot run migrations all the time. It has to be well-planned. It's just how it is. We still have some any tools and we want to get rid of them, but when is a good question.
Matt: 35:31 Those are harder to track as well. You can't necessarily write a bot to figure out where things are not typed correctly, especially with things as complicated as forwardRef and all that stuff.
35:46 I've got a couple more questions then about at what point in the migration, because you started, I think, a few months into the migration, at what point in the migration did your engineers start going, "Oh, this is fantastic"?
36:01 When did you start getting the benefits of TypeScript? How long did it take and has that continued? Do you still feel like you're getting more benefits the more type-safe you make the application?
Priscila: 36:12 Oh, I see this happens until today. Yesterday, I was checking Slack, our frontend channel, and Evan, one of our colleagues, he wrote "I love TypeScript because of this." This happens all the time, and it's just amazing.
36:32 You noticed that, in our codebase, we still have some .js or .jsx files. I can guarantee you that most of them are our tests, unit tests. Still, I am dreaming of converting them to TypeScript, is one of my personal goals.
Matt: 37:38 When you do write tests in TypeScript, writing tests in TypeScript can be a little bit annoying, especially when you have mocks that you want to only pass a partial of or things like that. How do you get around that? What techniques do you use?
Priscila: 37:52 This is one of the reason why some folks, they were against of converting tests to TypeScript because usually they don't want it to pass all the props that a component requires, but only the ones that they want to test. I noticed that some tests we had in the past, they were wrong because of the props we were passing or because of some missing props.
38:23 I really saw that TypeScript was helping me with that. I brought this to the frontend TSC meeting. Many folks liked the idea and they are doing the same. What I do to get around them is, I don't get around them.
38:43 I mock most of my tests. Maybe not everything, but what is required, I mock. I try to write some dumb value. For the majority of these things, we have also...I forgot how it's called. It's just like a file where we have an example of how this API responds. Then we reuse this.
Matt: 39:20 Oh, a fixture.
Priscila: 39:22 Exactly, a fixture, we have many fixtures. Usually, it's annoying, but it's something annoying that I don't mind because I really wanted to write a good test, so I do that.
Matt: 39:41 You just got to eat your vegetables basically. You've just got to do what TypeScript tells you, instead of putting an "as" on it or something, or trying to get around TypeScript.
Priscila: 39:49 No, I don't get to try around TypeScript, only when it's really my last option.
Matt: 39:59 Only for forwardRef.
Priscila: 40:00 Yeah.
Matt: 40:03 Let me see if I've got any more questions for you. I think we talked a little bit about React and TypeScript. I actually know. Let's go somewhere else, which is, on the team, you mentioned that you sometimes need to do some more advanced TypeScript.
40:22 Does the team, in general, get stuck on complicated TypeScript issues and errors? What's the process around that? How do you solve that as a team? What problems has TypeScript brought with it since you adopted it?
40:44 I know there were about four different questions in there. I figured there might be one of them...
Priscila: 40:49 Sometimes we have some complex types, but most of the time they are simple. I'm trying to remember a complex type right now.
Matt: 41:06 What might be a good one actually is to circle back to the component library stuff. Is there particular considerations that you have when you're writing types for a React component library that's different for when you're writing application types?
Priscila: 41:22 Yeah, like the ones we use usually in a component library they are these primitive types. The most simple ones is string, a boolean.
41:35 When I'm writing types for a viewer page or something more complex, a group of components, then I have to think about what the API is giving me. All these objects, all these things, and it can vary according to the parameters you are passing, you are informing the API.
42:00 We don't have this end-to-end type checking yet. It's a dream, but I think we are far from that. All the types we have, we have to define them in the front-end and sometimes it can vary.
Matt: 42:26 Because your backend is written in Python, so even though I guess most of it is in the same repository, you're not able to generate code or generate types from it. You have to write the types yourself in the front-end to try and get around that.
Priscila: 42:41 Exactly. One thing that I can think now is the type of the event, for example, that I receive, it can be, so we have type stack trace, we have type exception, we have different types of events that you can have at the same tree.
42:59 We have to define these different types for each of these cases, and they are used in one place. We have to check, it has this property, then do this, or things like that. They are a little complex.
Matt: 43:15 I can imagine. Priscila, thank you so much for talking about this. This was so cool to hear how a company like Sentry does so much with only 12 engineers working on the TypeScript migration. I imagine it feels amazing to have done it and to have completed the TypeScript migration and now be living in a fully TypeScript world. Thank you so much for joining me.
Priscila: 43:38 No problem. You are welcome. I enjoyed so much. I actually wanted to keep talking.
Matt: 43:45 Nice, I'm sorry we got to finish there.
Priscila: 43:46 No problem. Until next time then, bye-bye.
Matt: 43:49 Sounds good, bye.