From the course: DJ Patil: Ask Me Anything

How did you use data in grad school?

From the course: DJ Patil: Ask Me Anything

Start my 1-month free trial

How did you use data in grad school?

“

(uptempo music) - When I was a graduate student as I was kinda going through this process of trying to think about what problems I wanna work on in grad school, one of the things that I was just real excited about was this idea that people were putting data on the web, and so I was kinda just looking for problems, and I found that the National Weather Service was posting a lot of their data. And these were the early days. There wasn't any cloud services where you could easily download stuff. And so I'd started to download a little bit of the data and just try to see if I could graph it and find patterns in this data. It required so many computers that I didn't have, what I figured out that I could do was that if I effectively leveraged what might be called a security breach or a loophole (laughing) in the system, I could take over everyone's computer in the department. But if they were on the computer, they would see it getting slow. So I just started, what I'd do is I'd come into the math department at about eight or 9 p.m. at night and then work until about four or five in the morning, and I would just take over every machine, and I'd download all the data I could every day from the National Weather Service, try to find patterns and then produce a pattern, produce an image that I could then walk around the meteorology department and say does this look like something interesting or not? And then I would just go around, and they would just say nah. And eventually I came across this pattern that people were like whoa, that's kinda interesting. And so as a result, that led us down this question of like wow, maybe there is a new way that we could characterize the quality of forecasts and the uncertainty in forecasts. But fundamentally what that was is I was doing a data science exercise. I was finding data. I was pulling it down. I was going through it. I found the people who actually own the data, and I was asking them and interacting with them to say does this have context, and they were teach me, I was teaching them, and together, we were able to form a collaboration. (uptempo music)

Contents