Today, I speak to Karen Hebert-Maccaro, Chief Learning Experience Officer at O’Reilly Media. Supporting over 2.5 million technical specialists worldwide, the complexity of what Karen manages is absolutely mind-blowing. This episode is useful to anyone who is looking for new ways to personalize learning at scale.
About O’Reilly Media
O’Reilly is a learning company that helps individuals, teams, and enterprises build skills to succeed in a world defined by technology-driven transformation. From in-person conferences and live online training courses to self-directed learning and immediate access to problem-solving online, O’Reilly cover a wide range of digital learning experiences.
Listen to this episode now
Subscribe to Learning at Large:
Or search for “Learning at Large” on your podcast player of choice.
Simon: [1:16] Hello, Karen, how are you doing?
Karen: [1:18] Simon, how are you?
Simon: [1:19] Very well, thank you. Welcome to the show. Just to get started, can you introduce yourself and tell us a bit about your role at O’Reilly?
Karen: [1:27] Of course. Thanks for having me. My name is Karen Hebert-Maccaro, and I am O’Reilly’s chief learning experience officer. I joined O’Reilly about two years ago. In this capacity, I oversee our learning strategy for O’Reilly online learning, which is our learning platform, and also learning and training events that happen beyond our platform. We do live events associated with our conferences; we also do them in standalone situations on occasion. [1:57] We also work with individual clients to build custom experiences for them. So my role and responsibility is to ensure that we’re designing, delivering, measuring the best learning experiences across the entire O’Reilly portfolio.
Simon: [2:13] Thank you, Karen, that’s brilliant. Can you tell us more around the kind of learning platform you’re developing and the work you’re doing there to change the way you’re supporting people with learning?
Karen: [2:23] Sure. So, O’Reilly’s online learning platform is truly multi-modal. We have experiences that can be consumed by the learner in a variety of formats, everything from what I call performance-adjacent learning—which is learning that happens in the moment of need, where someone has to solve a problem or get an idea and then jump back into their work—but also learning that is premeditated, if you will, that might take a bit more concentrated time and be scheduled into somebody’s workday or weekend as they see fit.
[3:03] Live online training is something that was brought to the platform over two years ago now in an attempt to create a few things. The first is to create a concentrated period where somebody could set aside to come and engage in a very active experience through the platform—so you don’t have to travel anywhere, it’s available to you wherever you have access to the Internet and the platform. But it gives you direct access to people who are also interested in whatever that topical area is, as well as to the expert that O’Reilly has brought in to actually deliver that experience. [3:42] They range from one hour to several hours , and they provide a synchronous way to engage on the platform that complements our asynchronous experiences, which are sort of on-demand at that moment of need. They’ve been wildly successful, but one of the things that I think makes them so successful is the fact that when you are engaging on an online learning platform, you can get access to a whole wealth of knowledge and information regardless of what platform you’re using.
[4:19] But when you get into a situation where you have a specific question or you aren’t sure how to interpret something, getting some sort of support—live or asynchronous support becomes critical. Our clients really enjoy the opportunity to engage with our experts, to ask questions, to be involved in discussions to get more active in their learning. So, it’s a nice complement to the asynchronous work on the platform.
Simon: [4:41] You’re supporting 2.5 million learners through that platform—who is the audience, and what makes them different to other audiences? Because you are supporting a very particular need in the marketplace, aren’t you?
Karen: [4:51] Yeah, we are. I can tell you a few things that will be useful in terms of understanding the O’Reilly audience. We have accounts and corporations with the Fortune 500, the Fortune 1000. We have representation across every industry vertical you can imagine, from banking and finance to obvious high-tech. But one of the things that is really interesting about O’Reilly is that Tim O’Reilly, the founder of the company, started the organization 40+ years ago. And at the time, he was focusing his mission around spreading the knowledge of innovators, specifically through the medium of books. [5:35] The primary focus of those books in the early days was technologies that were being utilized inside corporations at that time or emerging as becoming more important to the business at that time. So we have this long history of essentially being focused on understanding in deep ways the technology sector. We helped software developers learn to code in every major language, Tim O’Reilly’s been credited with coining the open source software movement with Web 2.0.
[6:05] So, we have a large percentage of our current learner base who are in technical roles and who are using our product as a way of facilitating the continuous development of their technical skills. That is, and has always been, at the heart of O’Reilly. But as I mentioned, given the breadth of both industry and size of company that we engage with, plus we have a B-C audience; we have individuals across every vertical inside a company you can imagine. We do have individuals who are in sales roles, who are in marketing roles and G&A sort of functional support roles for finance and the like. So, our content is pretty broad, but I would say the majority of our learners are in technical roles or are attempting to learn technical skills.
Simon: [6:58] Thank you. I mean, actually, from my own background twenty years ago moving into digital I used your books to kind of find my way, because that industry is kind of self-taught.
Karen: [7:11] Your story is very common; I can’t tell you how often, when I’m traveling and talking with individuals, they say, “Oh, I learned how to code,” or “I learned web development,” or “I learned…” You know, it was the book. “I remember it had the animal on the front…” and they’d tell me what animal it was. It’s a nice history to have as a company.
Simon: [7:31] It’s interesting to think how my own kind of learning has evolved through that time as well, and how I get the answers and support I need in what has become only an accelerating technologically advanced world we live in. What kind of trends are you seeing now that are really pushing you to respond to the audience and their needs as highly technically skilled people, in some cases, to give them what they need?
Karen: [7:59] It’s a great question. We are, at O’Reilly, spending a fair amount of time thinking about the way the learners on our platform behave. In fact, so much of our time is being spent there that we’ve actually begun to make product enhancements and add features to the product for our customers to help them track the behavior of learners on our platform. So their learners, their corporate employees, who may be on our platform. [8:25] And when I talk about learning behaviors, right now we’re primarily talking about four types of behaviors on the platform en masse. Linear behavior, nonlinear behavior, broadening behavior, and deepening behavior.
[8:38] I’ll go through each in turn very quickly, but the reason these four behaviors are so interesting to us is that they tell us something about the needs of the learner based on what sort of behavior they’re enacting in the platform. So, the linear learner is going in a more sequential way, let’s say consuming content in the book from chapter 1 to chapter 2, to chapter 3. It’s a learning path where they may be going in order and consuming all the various multi-modal content that would be in that path in the way that it had been designed in sequence.
[9:14] And then, nonlinear behavior is what I call performance-adjacent behavior, and that’s behavior that isn’t as consistent across the sequential order that the content has been developed in. Rather, the individual seems to be—in the examples I gave you—maybe jumping right into chapter 3 of the book, skipping 1 and 2. And maybe they’re even going to the fifth or sixth page in chapter 3 and reading page 6 through page 10 of chapter 3. They may go into a video course, but they may skip the entire first module and go right into the part of the video course that’s of particular interest to them, and that might be two-thirds of the way through.
[10:02] Part of what we see with the linear and nonlinear behavior is that it is at least partially driven by the proficiency of the learner. So, when learners are new to a topic, they tend to prefer a higher degree of structure with the smaller amount of content. Once they reach what O’Reilly calls the line of structural literacy, which is a certain amount of fluency with the topic—an understanding of its relationship to some of its adjacent topical areas, if you will—they begin to behave in that nonlinear way. [10:33] So we found this to be particularly interesting when we began to think about how to design a product to meet the needs of both the early proficiency users and the more advanced users when it came to the way they were going to behave. [10:51] Broadening and deepening is just a way of us tracking whether or not a user or learner on our platform is really focusing on a particular area. For us, that’s defined by a variety of metrics—but things like if they read multiple books on a particular topic or they engage a live online training and they do a learning path and read a book that’s all related to, let’s say, learning how to code in Python. Then, that becomes a deepening area for them. We can understand that that’s a core focus where they’re attempting to really advance themselves.
[11:28] Broadening is a behavior pattern on the platform where individuals are maybe looking at more than one topic. Very often, those topical areas are adjacencies—so, for example, an individual might be gaining access to experiences on the platform around agile development methodologies. And then you might see that they go and look at lean processes or you might see them looking at project management 101. There’s an argument to be made that those things are all somewhat related. The individual is beginning to understand the connections between agile and lean, and agile and basic project management. That is a broadening behavior.
[12:14] One of the things that does happen sometimes with broadening is we’re able to say, “Oh, look”—if you look at an entire company’s worth of behavior and you see that a whole bunch of individuals are beginning to look at an area that has been adjacent to where they’ve been focusing on the past, it may be an indicator that that’s going to become an area of focus. So, it’s a way of almost getting some early indication of where your learner base is going.
I know that’s a lot, but I thought I’d walk through those four areas of behavior because we’ve been spending so much time thinking about them, and then building to accommodate the differences in need of the learner, depending on which behavior they’re exhibiting.
Simon: [12:56] That’s really interesting. The complexity of what you’re managing here is—it goes into so many different directions, then. You’ve got multiple modalities—I mean, from what I understand it’s a full range of live learning, books, online resources, videos… it’s a whole lot, isn’t it? Plus you’re dealing with technology, which is changing every day. How do you keep pace with the demands of the audience and the changes in the market to keep the content relevant and supporting people who are in some cases at the cutting edge?
Karen: [13:29] That’s a great question, Simon. I actually want to mention something about what you said about sort of the fiercely complex situation in which we find ourselves and being able to be nuanced in the way we talk about measuring and learning from the data that the platform produces. We are believers that that kind of nuancing and complexity is actually long overdue in the world of learning. [13:53] There has been an overreliance—and this is not being critical, I’ve been a chief learning officer inside corporations and a head of talent and I understand the unique challenges that are inherent in roles such as that—but one of the things that we have been a little too lax on is the overreliance on very simplistic metrics to understand our learners and whether or not we’re being successful.
[14:24] It’s easy to measure things like hours engaged in learning, or completion. Those things are not inherently bad, but they are inherently limited. One of the things that worries me about measuring things such as hours spent in learning products or platforms or classes or experiences or completion data, is that that really tells you a very small amount of information about the value that’s being derived or the impact that that experience is having. It may tell you very little, it may tell you nothing about what really matters, which is are people finding what they need, getting support to do their jobs better, are they getting back into their work and doing it with greater ease, with more productivity, with better outcomes? [15:26] That’s what we really want to understand. So, we are focused on being more nuanced and accepting and embracing the complexity of human behavior when it comes to learning in hopes of getting us closer to the space around truly measuring impact and not just measuring things like compliance. I just wanted to mention that because I think that’s sort of part of our philosophy and I think it’s important.
Simon: [15:53] And I’m totally with you on that on measuring impact and not compliance.
Karen: [15:56] And to answer your question about how do we keep up with the unbelievable pace of technological change, how do we meet the needs of those that are cutting edge in the technological space? Actually, we have an entire editorial team as a result of our early days as a book publisher—and of course, we still produce books today, but it’s no longer the sole method of providing—spreading the knowledge of the innovators in our network. But we have a whole team of editorial professionals who are intensely connected in the markets around technology and business. [16:37] We are providers of a dozen conferences annually across the world, where we are interacting with our conference chairs and presenters who are cutting-edge practitioners in their respective fields. So we build our network, in part, by the fact that we are a convener of expertise in a variety of different forms.
[17:00] So that’s one way that we stay in touch with what’s happening literally inside every major organization, every small startup, is by really becoming a center of professional work. We also have a separate team within O’Reilly called O’Reilly Radar, and a gentleman named Roger Magoulas is the VP for Radar for O’Reilly. And that Radar team is focused on not necessarily what’s happening today inside of companies and inside the technical world, but what’s emerging today that is likely to become more mainstream in, let’s say, two or three or five years from now.
[17:38] They’re bringing those ideas to our partners and our learners and our network. We have something that we call FOO, Friends of O’Reilly, and we do… so we have a machine that’s been built since the earliest days of the company, and it has multiple components but it’s really about leveraging this massive network we’ve built and the convening power of our conferences and events, as well as the thought research and leadership of our internal Radar and editorial teams to stay absolutely on top of what’s happening today and what’s coming next in the world of tech and business.
Simon: [18:18] That’s really great. Brilliant. Okay, just to wrap up, can you recommend a book to other learning leaders that’s really inspired you or progressed your own understanding of how to grapple with the challenges of scale and learning at scale?
Karen: [18:31] Oh, that’s a really great question. Actually a book called The Surprising Truth About How We Learn and Why It Happens. It was written, I believe, maybe four years ago by Benedict Carey. What I love about it is it makes cognitive science, learning science, really accessible. It talks about what it is we grow up learning about learning—you know, self-discipline and all that—but it also goes beyond that to talk through all sorts of, many many years and decades, of learning and education research and studies that uncover how our brains actually retain information and make it accessible to everybody and practical in terms of good learning design. [19:27] So, that book jumps to mind right away just because I’ve always felt that it was a really good basic primer of understanding of how the brain works and how we can translate that into good, solid learning design.
Simon: [19:42] Thank you, Karen. Brilliant. And where can people go to find out more about you? I’ve read some of your brilliant whitepapers, which I’d recommend to people listening here, and you’ve talked about performance-adjacent learning and your interest in that. But there’s a number of subjects that you develop research around and talk about in a very compelling way. Where can people go to find out more about you and to find out more about the work you’re doing?
Karen: [20:05] Yes, thanks for asking. Of course, I’m happy to have individuals following me personally on LinkedIn; they can just look up my name Karen Hebert-Maccaro. I have a site at O’Reilly that you can look up—search my name in the OReilly.com site and you’ll find all the writing I’ve done, including the whitepapers you just mentioned, and then others like articles and blog posts that I’ve done. Lastly, I’m on Twitter @learnadjacent, so people can certainly follow me, engage with me there. But I’d welcome reach outs and connections from your listeners; it’s always fun to build the community and network of learning pros and people interested in the fate of learning. So, I’d welcome all that.
Simon: [20:48] Brilliant, Karen. Well, everyone listening, please take Karen up on that offer and keep this conversation going. Karen, thank you so much for your time today—it’s been really interesting to hear your point of view. Really inspiring conversation—thank you so much for making the time.
Karen: [21:03] It was my pleasure. Thank you, Simon.
Simon: [21:11] Thanks to Karen for some brilliant insights into how to support the needs of different learners. Please do follow her on LinkedIn and look up her whitepapers, which offer brand-new research perspectives across a range of subjects—including ROI, measurement, leadership, and learning behavior.
Join the conversation!
We really want to hear your thoughts on today’s chat. Please join us on Twitter to keep the conversation going—just look us up @elucidat. You can also email me, email@example.com, with thoughts, suggestions, and questions. Finally, don’t forget to subscribe to Learning at Large in your favorite podcast app. And if you’ve liked what you’ve heard, please give it a great rating. Thank you so much, and until next time.
Latest posts by Simon Greany (see all)
- Ep10: Socializing your learning design to support global audiences - September 17, 2019
- Ep9: Up-skilling your workforce to future-proof your organization - September 3, 2019
- Ep8: Supporting the learning demands of 2.5 million technical specialists - August 20, 2019