Rethinking traditional approaches to release notes -- recording of WTD Australia presentation

by Tom Johnson on Feb 17, 2025
categories: ai • api-doc • wtd-podcasts • writing • podcasts

• Description
• Article version
• Slides
• Transcript

Description

Here’s the presentation title and description:

Rethinking traditional approaches to release notes: A new method using AI and file diffs

While exploring use cases for AI in writing documentation, Tom found a method that fundamentally changed one of his core documentation processes: generating release notes from file diffs. Most of the time, writing release notes meant tracking down engineers to understand upcoming changes.

He realized he could skip much of that work and find most changes himself by analyzing file diffs with AI tools.

This practice led to several key insights:

• Engineers aren’t telling technical writers half of the changes they’re making. This contributes to documentation drift and outdated content.
• Analyzing file diffs expands my influence within the release process. He now possesses a broader view of the release than the product’s technical leads, increasing his value in the release process.
• AI tools are good at translating file diffs into clear information. This makes file diffs a valuable resource, not just for release notes, but also for documenting any changes engineers make.

In this presentation, Tom will show his approach to creating release notes using file diffs and AI. You’ll learn how to use AI tools to change your approach to documentation.

Article version

Here’s the article version of this presentation: Using file diffs for better release notes in reference docs.

Slides

Slides are here:

Transcript

Here’s a transcript of the presentation (cleaned up with AI).

At work, you might have a lot of people who are interested in AI. They meet together to share tips, tricks, and what they’re learning. I run a group like that at my work, and there are probably half a dozen similar groups spread across different pockets of writers. So, I’m always looking for different things. I love to experiment; it’s one of my favorite things. I like to try something new, and this is one of those techniques that I tried once, and it stuck. It just made writing release notes so much easier. It gave me a lot more control. I felt like I could completely put a stop to documentation drift. It started to change the relationship I had with engineers. They began to look to me for, like, a voice of authority about what was actually in the release.

I could correct them and say, “Hey, did you realize you accidentally pushed this method in the current release when you intended it for the next one?” or “Oh hey, you’ve got your method names mixed up,” or something. You know, it gave me a lot more insight into what was going on. I wasn’t really dependent on the engineers for all the information. So, I’m going to share just this technique, and if you have questions or thoughts, jump in. I will try to… I can’t actually see the chat in my little screen. I’m sure there’s some little widget that I can toggle. But anyway, if you have thoughts, please jump in at the time, and I’ll pause periodically to kind of surface conversation.

Let’s get going with a bit of an agenda here. So, I’m going to talk about how the release notes process typically goes, the new process using AI with release notes, and then some risks, trade-offs, advantages, and discussion. I don’t have, like, 200 slides to get through or anything. It’s fairly minimal, but it has been a profound shift in how I operate.

Okay, so typically, the release notes process goes like this: The release team has some kind of launch entry. They try to get input from engineers and so on. They’re indicating what features are going in. I mean, most software shops have some kind of launch tool that tries to capture the big rocks of the release. But invariably, the tech writer has to reach out to engineers. Maybe you have a staging doc, and you’re like, “Hey, what’s going in the release? Any new features? Any bug fixes? Any changes?” And you might ping one engineer who has their view of the product, and another who has their view, and so on. You might have to coordinate with different PMs and so on. It’s a lot of pinging people and trying to crowbar information and get it added to a doc.

When they do add it to a doc, it’s often opaque or confusing, or just thrown out there in a quick brain dump. You have to sort it out and figure out: What does it mean? Is this significant? Is this insignificant? You’ve got lots of back and forth, and so on. I mean, it can be a real hassle. And the release processes now aren’t just taking place quarterly. Every API I support has either a bi-weekly or weekly release. They’re always pushing stuff out, so this release cycle repeats itself very quickly. If you support multiple products, you might be constantly writing release notes, constantly having to gather information from engineers and try to make sense of it. Well, it can be a little bit exhausting to do that.

And, it’s not very accurate, to be honest. I think a lot of things slip by. I honestly believe that engineers aren’t telling tech writers about a lot of the stuff that’s going out, and it’s creating this documentation drift that is making the documentation less valuable.

So, the new process that I’ve been following is to use AI with release notes. The key idea is that most of the changes in a release are evident in a file diff. And if you can use AI to easily read that diff and get a summary of what those changes are, you’re now empowered to easily write release notes. I mean, it’s as simple as that.

Okay, so I’m going to break it down into more detail here. This is kind of the detailed process. First (and I’m going to talk through each of these in more detail), I build the API reference output. This is like the regeneration of the API documentation – your Javadoc or whatever reference-type documentation you have. I then get file diffs for each file changed in that regenerated reference documentation. Then, I use AI to get readable summaries for each of the file diffs. I use AI (or sometimes just do it manually) to organize those readable summaries in a way that makes sense – edit, restructure, analyze, fix. Then, I link every code element that’s mentioned to the actual reference page entry, just to make sure that it’s real, that it’s legit. I check the updates again, like preview versus what’s currently out there. And then finally, I review it with engineers. So that’s a snapshot. Now let’s go through each of these individually.

Okay, building the API reference output. I love this step. Okay, this is where you run a command, and your documentation generator (whatever it may be – it could be Javadoc if you’re working with Java files, it could be any number of documentation generators) kicks off. They run through the source, and they generate the output, and it becomes the reference. Well, if you build the reference output, or if you just simply get a hold of the change list – the list of commits and so on of all the changed files that somebody else ran to build it – that is going to be key.

It allows you to really take control of the build process. Because even though, let’s see… yeah, even though you could be working in any number of languages, the reference documentation is invariably going to be in HTML or Markdown. And because of that, it’s kind of like the key resource to have. It’s very rare that an engineer can make a change to anything in the source that isn’t reflected in the API reference documentation. I mean, any meaningful change to, like, an outside developer, is going to come out in the reference output. So, having control of the API reference build is key. Some tech writers don’t like to do this, but I think it’s pivotal to being able to have all these file diffs at your disposal. You’ll know when the last reference was built versus the current one, and so on.

Okay, so now that you’ve got the regenerated reference, the next step is to get file diffs for each of the files changed in the regenerated reference. Your version control system probably has some form of, like, git show, and you put the commit hash and a file name, and it tells you exactly what the file diff – the differential between the previous state and the current state – is for that file. And it will put little pluses for lines that were added, minuses for lines that were taken away. And, yeah, that’s kind of like the key resource here. I know I said that about the previous one, but this is even more important. Having this list of changes for each of the files is the gold mine for having the evidence of what’s changed.

Um, yeah, we don’t actually… In my group, we don’t actually use Git. There’s some other version control system, so it’s got its own set of commands, but it’s very similar to this. Okay, now that you have a bunch of file diffs that tell you the pluses and minuses (which are going to be hard to read), I mean, a lot of times, version control systems have a nice graphical user interface that shows, you know, what you’ve removed in red and added in green, but it can get very confusing in different shades of red and green, and so on. And if you’ve got something that’s complicated, with lots of things added and removed, AI does a pretty good job at really parsing through those files.

And here, you actually have a choice. You can either kind of get all the changes for every file that’s been changed in the regenerated reference, or you can go file by file if you really want to zero in on the changes in a granular way and kind of sort through it. I often like to go file by file. I feel like I might miss something, and I want to be able to go slow to analyze what the changes are. But yeah, basically, when I rebuild the reference documentation, I’ll probably have like a dozen files that have changed. And then with each of those files, if you run a command like git show and then the commit hash and then the file name, it’s going to give you all this – the file diffs for that. And you pass that file – just copy and paste it – into your favorite AI tool that you’re allowed to use.

You know, any of these modern AI tools from Gemini, Claude, ChatGPT, and many others that are capable, and which you might, you know, be greenlit by your company’s policy to actually use, can do a really good job at reading all of the changes in that file diff and giving a human-understandable explanation of what’s changed. You can have a really sophisticated… Let me show you. Like, you can have a very sophisticated version of this. I’ve got a couple. Like, I once put together a really elaborate prompt to try to go through this, where I said, “Hey, you’re a tech writer, here’s your target audience, and here’s how to interpret the file diff. This is what a plus means,” and so on. But this, it turns out, this is really unnecessary. Like, it’s kind of like the AI tool usually understands how to read the file diff natively. They don’t need an explanation.

You can simply say… Let me see. I think I have a shorter prompt before this. You can simply say, “The following is a file diff from regenerated reference documentation for an API. Provide a human-readable summary of the file diff, noting in granular detail everything that has changed.” So, something that simple works just as well as the most elaborate prompt you can concoct. Okay.

All right, let me pause briefly right here because I’ve been talking for a little bit. Does anybody have any questions or comments about just, like, getting file diffs for your reference documentation and using that as a starting point for understanding, like, what’s been changed?

Yeah, there’s a couple of questions on the chat, Tom. So Helen asks (I think you probably answered that). I got it, yeah.

Okay, so Helen says, “Is there a particular prompt that…” Yeah, got that one.

Yeah, honestly, just simple prompts like, “Give me a human-readable version of the changes in this file diff.”

Just passing the diff files through AI represents a risk in security?

Now, this is where I did take careful note to say, make sure you use the tools that your company allows you to use. You know, like, at our… at my company, there are certain AI tools that actually work amazingly, but we’re not allowed to use them. So the ones that we are allowed to use have a little bit more, um, I don’t know, workflows, controls, and so on, but they’re still really good. They’ve gotten really good in the last, like, two weeks, to be honest.

Um, let’s see. “Did you do any of this diff analysis prior to AI being available? If so, what are the improvements you identified in productivity?”

By… No, you know, prior, when I did release notes, yeah, I mean, I would look at the GUI, the UI, and kind of get a sense of, maybe, like, “Oh, looks like a big chunk of something was added here and there,” but I was much more reliant on engineers to kind of talk through it and kind of be, like, running the show. So yeah, I might just be in a very release-heavy moment, too. The group I’m in really pushes out a lot of different updates. They’re kind of in an active building phase for one of the products, so they’re co-building it with partners, and that’s kind of how they go. And so there’s lots of releases. But, uh, yeah, yeah.

Okay, let’s keep going. All right, step four. Now, if you go file by file, if you had a dozen files, and they were just listed alphabetically, and you asked AI for readable summaries of each file, you’re going to have a lot of information. Some of it might be redundant. Maybe your reference docs have, like, a REST version and an RPC version (that’s our case). So, maybe you’re seeing the changes listed twice. But generally, you want to now start grouping things. It’s kind of like you have a pile of laundry, and you want to put all the socks in one pile, all the shirts in another pile, all the pants in another pile, right? You’re grouping like information together.

You can use AI to do this. You can give it a prompt like, “Hey, group like information together in a logical way,” or you can just do it yourself, because maybe you look and you see, “Oh, that’s a change, but nobody’s going to care about that change,” or, you know, “That’s just some test unit test code that nobody even uses. I don’t want to draw attention to it by having AI try to, like, fit it into the release notes.” Or maybe it’s just a typo. This is something that, um, I don’t think really needs to be mentioned in release notes. Somebody fixed a little grammar issue. They fixed a link. Sometimes it can confuse the AI. The AI thinks, “Oh God, something has changed,” when really it’s just trivial.

So, a little bit of human intelligence here to organize things if you want. But I mean, at this point, you’ve sort of gathered the information, and so you’ve… you’ve done the hard work. It’s just shaping it. And here is where you would also want to refer back to maybe your launch entries that indicate what the main features are. That will give you a sense of hierarchy, like, “Oh, okay, so this feature X is the big new feature. Yeah, we probably want to trumpet that more and put that up at the top,” and so on.

This is sort of a continuation of the organization, one where you’re just going to want to, like, edit, restructure, analyze, and fix a lot of things, right? There’s no… By no means am I trying to suggest that AI is just a magic button, and you press it and voila, you’re done. It’s just a way to sort of make sense of all the file diffs and give you that information so that you’ve got all the ingredients on the counter, and you’re ready to start, you know, cooking, like you want. It really just makes all the information available, and you can see it, but you still want to have control. You’re still steering the ship there. I’ll try to get as many analogies as I can.

All right, now, step number six. This is, um, this is a step that I, uh, feel pretty passionate about. And maybe this was something that was more problematic earlier on in the AI days, but I like to link every code element to its corresponding reference page. And usually, it’s not just a page; it’s like an anchor link on that page, or even in a table, it’s an anchor link in the table. And I do this for two reasons. One is, I want to make sure that the elements haven’t been hallucinated. I also want to make sure that when I am doing my own QA on the content, that I can verify it. If it says, “Feature Z was added,” I want to be able to click feature Z and kind of compare it against the preview versus release and see that, “Oh yeah, feature Z was actually added.” And if you’ve got everything linked, it’s going to make that much easier.

It’s also very easy to, uh, get, like, casing mixed up. Maybe by not grabbing the link, it can be easy to maybe use snake case instead of camel case, or whatever it is, because you might have different, you know, versions of things. Like I told you, we have REST and RPC, and the casing varies, but we like to follow the REST casing, but the AI might give you the RPC casing, that kind of stuff. And when you link everything, it really, um, gives you more authority about how that casing is.

Now, you can actually… This step is not as tedious as it seems. If you’ve got a really powerful AI model that has a large token input of a million to two million tokens, what you can do is just copy and paste your entire reference material into your chat window of your AI tool and tell it, “Hey, uh, here are my release notes. Every code element mentioned in the release notes, please, uh, link based on this API reference.” And when I do that, it usually is about 70% accurate. Um, it really depends how large your API is. Uh, I actually have scripts that go and gather, like, all the different files and put them into one file so that I can easily copy and paste it in there. Uh, but it also… If you’ve got everything in one file, you can just go to that file and do Ctrl+F, and it’s pretty fast. At any rate, um, if you’re pushing out release notes every two weeks, you probably aren’t accumulating a ton of, uh, new changes every time. So maybe you have, like, at max, 10 elements or so that you’re linking. It’s not a huge lift to just do this manually.

Okay, all right, number seven. This is the QA step. Check the changes… Oh, actually, no, this is going beyond… Okay, so the previous step is where I mentioned checking the, like, preview against what your release… your current release is, to really verify the changes. But if you want to go the extra mile, what you want to do is check the changes in your release notes against the whole documentation corpus. If an element was deprecated or updated, how do you know if, uh, some code sample on some page that maybe you didn’t even work on needs to be updated? Well, now you have to scan these changes against your entire documentation corpus. Again, this is another job for an AI that can, uh, just chow down on millions of tokens and crunch through and actually identify needs. This is probably the step that is hardest and, and probably what leads to most documentation drift, because it’s a really big pain to go through every page of your documentation and try to see if the new changes you’re pushing out affect anything in the current version. Uh, but that’s, that’s where a lot of errors start to surface, and things get outdated, and so on.

But yeah, like I said, the, the AI tools that allow you to just paste in hundreds of thousands of words, uh, really are game-changers for this, because it’s, it’s not great to have to do this manually. You know, it really requires somebody who’s familiar with the, with the doc set. Um, and usually there’s lots of tech writers working on lots of different projects. You, maybe you inherited stuff you didn’t even know existed. So, again, depends on which tools you have available.

All right, then finally, send the CL to… or, sorry, why’d I put CL? That’s sort of a Googly term… uh, change list, your pull request, whatever it is, to your favorite engineers to review. And, uh, it’s always kind of fun when they just basically approve your changes and don’t have any kind of input or edits. It just looks good to them, and it’s like, “Wow, uh, like I hit the bullseye on this first shot.” Um, that’s, it’s kind of a little bit of a rush to do that.

Okay, those are the eight steps. Any, any thoughts, feedback? Let me check the chat here, see if we’ve got anything queued up.

All right. “I found out in 2017 or so, the engineers often don’t think changes, or often think changes don’t require docs when they do require docs, by doing that sort of diff check.” Yeah.

Um, yeah, you know, engineers… it’s weird. I’ve been a tech writer for 20 years, and still, um, like, every engineer that I’ve worked with, about, like, uh, 70% of them have never interacted with a technical writer. Uh, maybe they’re just younger or something. Yeah, they, they often don’t know what needs docs. Is it good enough just to have a little update in the reference? When do you need a code sample? Do you need a conceptual doc? Is that separate? When do you pull in tech writers? People are often in the dark. So if you, if, if you have a process that really relies on engineers to kind of steer you through it, it’s going to, I don’t know, take you through a lot of different ideas about how docs should be done.

So I, I love having control of things. It’s part of why I like this approach, is because it really puts the tech writer in control of the process. Um, you’ve got, like, you’ve gone through all the changes, and unless there are changes that are happening outside of, of the API, um, like maybe the data returned has gotten a lot better, they fixed something, maybe they started populating data you didn’t know about, you know, but by and large, when you, when you base all your release notes on the file diffs, you do know 95% of what has changed in the API, and you don’t have to rely on engineers to guide and direct, uh, what the release note should really be. There’s something nice about that.

“Tom, do you see any further ways of using AI to streamline this process, thinking about how generative tech is improving all the time?”

Yeah, you know, the most obvious way is, once people come out with the agentic workflows, it seems very easy to string together a list of commands that would all be sequential and could run, you know. Um, like, for example, coming back to my process – that’s a great question, by the way – um, “Get the file diffs for each file changed in the regenerated reference.” Well, in order to do this, uh, there’s another command that I usually run that says, “What are all the changed files in this change list?” Right? So that’s one command, and I want to take the, the result of that command and plug it into the next one to get the file diffs.

So I would love to build a script that basically goes through each one of these steps as much as possible, at least steps one through three, and, and really tries to just stack up the commands one after the other. Um, and that’s when the agentic workflows are available, it’s going to be another game-changer, because it’ll allow, it will allow you to break down any complex process into little sub-components, modules that you can then execute. I mean, it’s like programming. You can program a workflow.

Another part that I actually, um, ah, oh, it’s… blanking on me. Uh, it’ll come back to me here in a minute. But yeah, there’s opportunities to really improve this in a lot of different places. Um, what was I gonna say? Oh, okay, now I remember. So, it depends what, what your, your reference generator is, but most reference generators also have a log output that tells you a bunch of interesting stuff that’s going on. Um, uh, Google’s, like, reference generation is a homegrown tool, so it probably doesn’t correlate, uh, with other tools, but it will tell me when an element is missing documentation. But that warning message is really buried in a, like, running log in the terminal. It’s very easy to overlook.

So I, I built some other scripts that would go and extract all those, like, “Hey, documentation is missing for this element,” uh, um, kind of warnings and put them in one little concise file and allow me to easily create a bug. And previously, I had, had tried assigning the bug to engineers to say, “Hey, you’re missing some, some documentation from your reference files,” and about half fixed them, and the other half didn’t. And I finally just went in there last Friday, actually, and I, and I just pasted in the source file into our AI tool and said, “Hey, uh, these are the warnings that we got, and this is a source file. Can you just populate the, the, um, elements that are missing descriptions with something?” And the ones that were missing descriptions were very, like, very basic, which is why they’re missing, because the engineers thought, “Ah, it’s one of those really obvious things.” Like, uh, you know, let’s say you’re documenting a UI that was, like, “Your address,” and you’re like, “Oh God, what do I put for ‘your address’ other than ‘your address’?” Something, you know? So it wasn’t, like, rocket science there.

Okay, um, but it was nice that it identified all the parts that were missing.

“Yeah, get AI to write the script.” Well, yeah, uh, that is another kind of part of this process, is, is “Build a reference API reference output.” I definitely used AI tools for this, because, um, yeah, there’s a lot, a lot, uh, that’s easy to automate there. Really, I don’t want to get too far into this, because everybody’s got different tools and workflows and so on. But yeah, this is, like, a huge strength. Um, I think something like half of the use cases for these AI tools are software-related right now. There was a, there was an article on Claude that people were passing around about how people are actually using AI tools, and software coding and writing were two of the most common use cases. So, definitely, you don’t have to be, uh, like a developer to put together a fairly sophisticated script. I’ve got a script that, that, um, like, there’s a bunch of different APIs in an SDK, so it cycles through each API, builds the reference output, and so on for everything. And, uh, yeah, it was pretty nice to just be able to, uh, put together something like that.

“Do I have a video that demonstrates this approach with yours, with some actual or fictitious samples, please?”

Um, uh, I do have an article, um, if you come to my site here, where it says “Tech Writer + AI,” and go to “Prompt Engineering” (which, by the way, I want to change that name at some point, so this might change), but, um, I do have an article here, “Using file diffs for better release notes and reference docs,” and I kind of walk through it. But do I have a video demo? No, because, like, um, I mean, all the docs… Oh, I do have sort of a video, but it’s not really, it’s more like a podcast. All the, all the… I mean, who writes release notes outside of work? And if you can’t share, like, your, your work artifacts due to firewall and security concerns, then, like, uh, it would be really weird to just go find an API and try to write release notes for something that I know nothing about. So, not really in a situation that’s very public that I can just, like, do that.

But I will say that of all the, the AI experiments that I’ve tried, the release notes process has been by far the most successful. Um, I’ve also got some colleagues who did this in other groups, um, at my work, and they were really blown away, too. They had similar processes, but they’re different tools, different toolchains entirely. They get repos and so on, but their release notes were suddenly so much more detailed and, and kind of just, like, accurate. Um, I, I realize that AI is not something that, like, in every writing scenario should be adopted. I know I’m an AI optimist, but I’ve really come to see different scenarios as being ideal for AI and non-ideal. And this is one of those ideal scenarios, because I don’t, I don’t sit around in my free time writing release notes, or even writing technical documentation. I mean, I kind of have some, uh, whatever, my course online and so on, but, uh, look, technical writing is not anybody’s, like, fun hobby that they do on the weekends. Every… If you, if you look at the traffic to any blog on technical writing, it drops off a cliff every weekend.

So it’s a day job. Creative pursuits, on the other hand – you’re writing a personal essay, you’re writing fiction, you’re writing, um, just a blog post – that’s the part where you really want to engage your brain, because otherwise, there’s no value for the activity. It’s kind of like, uh, it’s kind of like music, to use another analogy, right? If you had AI, like, um, play music for you, it would remove the whole point and purpose of sort of playing music, which is probably a deeply kind of… I don’t… Not a musician, but I imagine it’s something that, uh, you feel, and it’s, like, rewarding, you know? It’s not really that way when you’re writing release notes. I, I, I’d rather just get it done, get it out of the way. It’s kind of like this, this dumb little art here, right? This is AI-generated, obviously, um, and, uh, because I’m not an artist, I don’t see a lot of value in sinking a ton of time into, like, creating this myself, in the same way that I, you know, just want to get the task done, get whatever info I need, and, and move on to the tasks that I want to do manually.

Okay, I, I don’t know where that came out of. That was a little bit of a tangent.

That’s all good.

Okay, “I’ve often found that release notes are a bit of hit and miss. Some software companies are really into them, while others are not that fussed.” I love that word, “fussed.” Okay. “I actually like release notes. They can be simple and not need a lot of texts.” Yeah, yeah.

Um, I, I sometimes get fussy people who are, who basically say, “Hey, look, I want to know every little thing that changes, like, no matter how small.” Like, “You want me to tell you when I fix a broken link?” They’re like, “No, no, but material, material changes.” And, uh, they want very granular details. I work for, like, APIs in the automotive industry, so you can imagine that people who are building, like, APIs into cars want to know when you change something. So they really appreciate the, the detail. Yeah, and, and I, I really get very specific. But if you’re in other scenarios, it might be overkill.

“Can that link to the article be posted in this chat?” Oh, thank you. It’s also in my slides, which I, I’ll post the link here, too.

“You’ve talked a bit about detecting syntax differences, say, missing docs, etc. Have you used AI to identify and document differences in API semantics, like the addition or removal of permission checks, and different things that happen in the background depending upon the parameters provided?” Uh, yeah, uh, not really. That’s a good point. Like, let’s say that they changed file permissions on certain things, and that’s not going to be evident, maybe, in the diff. Yeah, and that’s a great point that you bring up, because there are some things that aren’t included in, in the API reference documentation. I touched on it briefly, but let’s say you’ve got other scripts, let’s say you’ve got an SDK that has some other utility scripts that are updated, um, yeah, you’re probably not going to see those. Uh, and, and that’s a great segue to this next point that I really wanted to, to, um, to get into – the risks and trade-offs.

Um, let me jump around here a little bit. There’s, there’s a real risk of focusing just too much on the what instead of the why. Um, like, let’s say that a method was deprecated. Well, the AI tool is just going to tell you that this has been deprecated, but they won’t tell you why, because it, it doesn’t know. And, uh, for that rationale and reason, you have to go to the engineers and try to get it. And honestly, every time I ask them, like, “Why have we deprecated this?”, the engineers tell me an embarrassing, uh, story, like, “Oh, we coded it wrong, and we realized it, and we fixed it.” And it’s like, “Well, okay, what do I say?” Uh, but, uh, you know, if there’s a reason… It really depends on your product scenario. We work very much, like, um, one-on-one with a lot of partners implementing the API, so a lot of times, our partner engineers kind of provide this rationale, or the partners themselves are asking for the changes, so they don’t really need an explanation of the changes. They just want to know that it’s been done.

But if you have a public API that is, uh, used by millions of people, then you probably really do have to be more explicit and transparent about rationale. And that’s really just not going to come through in this process. You’ll have to really layer it in with conversations with engineers. And that’s, uh, brings me to another point. This is another sort of risk: There’s a sense of isolation and loss of rapport with engineers that comes about from this. Um, uh, like, if you’re able to just do this task mostly on your own, you don’t need to bother engineers to get detailed input. Uh, there is a risk that you’re just kind of, like, writing and doing it all, uh, in silence and on your own, with very minimal needs to, to check in with the engineers. You know, maybe their only input is to hit “LGTM” (“Looks good to me”) on the release note once you’re done with it. Whereas in the past, maybe you met with them in a room, and you kind of had a little bit of a chitchat, small talk, and you worked up to the, to, to the actual conversation, and, you know, it’s like… I mean, this is, this is the recurring theme of technology, is, uh, increasing isolation and disconnection with everybody around us. So, whatever, uh, it’s just another, another step towards the path we keep walking on, walking down. But yeah, this is a real thing.

Um, I feel like I used to have regular meetings with engineers where I would ask them, “What’s coming? You know, what’s, what are the changes?” And I’ve not had to do that. We’ve, we’ve just canceled the meetings, because it’s like, “Well, I don’t have anything,” and they’re like, “I don’t have anything to say.” And, yeah.

Um, there are a couple of advantages I really wanted to jump into, or a realization that I’ve, that I’ve come across that shocked me. Um, the first is that engineers, they don’t always know what’s being released. Um, I, I, I regularly meet with, like, tech leads, and they honestly don’t know what’s in the release. Um, they, they, or the product managers… I mean, they have a vague idea based on this launch entry and the big feature and so on, but a lot of the little stuff, they don’t know. And, uh, they really start to look at the tech writer as a, more of a sense of authority about, “Oh, you know, you’ve combed through, you’ve seen all the changes, and you’ve done this, this granular file diff. You tell us what’s in there.” Um, that changes the relationship a bit, so that the tech writers are no longer in a subordinate role, dependent on these godlike engineers for all their knowledge about the release.

Another realization is that I constantly see engineers slipping stuff in that they don’t tell me about – changing the definition of certain fields, you know, like, suddenly, you know, in a material way, in a way that’s actually like, “Oh, is that how it works?” Um, or, or adding other things in there, or deprecating little things. Uh, usually the major features, yeah, they don’t just, like, um, slip those in without some kind of, uh, announcement or doc bug. And usually, that’s part of a, a mature launch process, is that you do have, like, major features flagged in these launch entries that are, that are kind of verified, right? But a lot of little stuff, I feel like, slips on by.

And I’ve started to notice it. Um, and, uh, made me think, “God, maybe docs have such a bad reputation because for years, this has been going on, where engineers make changes and never tell tech writers, and there’s this growing drift between the documentation and the actual, like, API, that just happens because engineers never tell us.” And, and this new approach, where we can just look through the file diffs and get a very exact sense of what’s changed, can put a stop to that. No longer do these changes just slip by, and then we, we realize, “Oh, they got rid of that. When did they get rid of that?” So, um, I think that’s kind of a game-changer.

And then finally, I do think the value of the tech writer in the process does increase. I feel more of a high-value, um… One of the products, really, the, the PM wants to see the release notes before he will approve the release. And I’m like, “Wow, I mean, shouldn’t you be telling me what’s in the release?” Uh, that turning the ta- the tables like that is kind of fun. I feel more of an authority. I can, I, I’ve caught things, too. Like, I’ve caught things like, “Hey, you said that this method wasn’t going to be coming into the next release, but it’s actually in this release.” And they’re like, “Oh, really?” They have to cherry-pick in some changes. Or, um, my favorite one, last year, man, these… there was a team that was, like, they were very excited to, to release this end-of-the-year change, um, and I realized at the, the last minute that they, like, uh, got the method names mixed up between two different classes. And I was like, “Really, is this intentional?” And they’re like, “Oh, crap!” And they had to, like, swallow the release and fix it into the next year. And, uh, it was kind of fun to play that little QA role and just be like, you know, I, I, um, have more of a sense of, like, knowledge about every little change that’s going through here, and nothing really slips by me. So as a result of that, um, that, that last scenario, I decided to really start writing these release notes a lot earlier in the process, so give people, like, a week to make changes if I found anything, and they appreciate that.

Let’s see, we have any questions that have queued up? And, uh, oh, we got five new messages. Okay.

Um, “Deprecated because, because it, we replaced with a better alternative.” Yeah, coming back to that discussion about why they fixed it, it’s like, yeah, because they messed up. Uh, “Feels like developer teams are always enhancing or mitigating stuff without sharing.” Yeah, yeah. I mean, honestly, if you messed up, right? Like, let’s say you’re reading the docs, and you realize that there’s this embarrassing grammar issue, you want to just fix it. You don’t want to, like, call attention to it, right? And I think engineers must be the same way about code. They’re like, “Oh, how could I have written this a better way? Let me fix that.” And then they’re like, “Oh,” they don’t want to, like, call attention to their, to their mistakes.

“Have you considered automating any of this?” I feel like we already talked a little bit about that. Um, yeah, you know, this is a part where you can’t really automate: once you’ve got a draft of the release notes and you, you’re reviewing it with the engineers, there’s not a lot that you can automate. Although, I will say that, uh, uh, it’s actually, actually, that’s not true. So let’s say that an engineer writes back and says, “Hey, um, about this part of your release notes, XYZ,” whatever comment, you don’t have, have to actually think about the comment and interpret it and think, “Well, what would the change be?” You can actually just take your release notes, paste it in there, paste in the comment from the engineer, and say, “Hey, this is what an engineer said. Please fix it.” And like 99% of the time, it’s pretty good. Um, only if the comment is like, “Hey, you know, uh, rethink this in a complicated way,” where that might not, that might not work. But yeah, so could this be automated? Um, yeah, yeah, there’s a lot of just, like, “Hey, these are, this is mechanical, trivial stuff that they’re asking you to change,” and AI tools can just do this, and you don’t have to sit there and do it manually. It can really speed things through.

“I like this mention of the very important part of the tech writer’s job: creating and maintaining rapport with engineers, keeping communication going. I feel that this is a good thing that tech writers offer to an engineering team, helping the team maintain a healthy culture.” Yeah, um, just to kind of emphasize this a little bit, I honestly, uh, was, was thinking about, like, Friday, it was the day before a 3-day weekend, so a lot of people ski town on this day, but I was sitting in the office, and I was thinking, “God, I, I feel like I haven’t, I don’t talk to anybody anymore.” And that’s really sad. Uh, I mean, people invite me to lunch and all, but I’m, I’m kind of, like, I don’t often go. But, but just, like, we don’t often have these meetings that I used to have. I used to have, I used to have monthly meetings with every team I supported, where we’d talk through what was coming up on the horizon. Let’s talk through this big feature you’re putting together. And I really haven’t had to do that as much. Um, so, like I said, I do feel like, you know, this, like, another one of these tools that is making our lives easier but more isolated in the, at the same time. That might just be me in my situation, though.

“All right, with the hurdles you mentioned – isolation through improved tech, engineers not knowing what’s in the release – maybe it’s just a change in process terminology, process and terminology. The release notes prep meeting doesn’t go away but instead becomes a release notes chat meeting, complete with all your existing updates and a checklist for the devs, PMs, tech writers to go through.” Yeah, yeah, maybe, but then, then it’s like, do you need a meeting if it’s just, like, a little chat? Um, if, if, uh, people still don’t like meetings, um, and so if, like, you don’t really have anything substantial to work through, it might not be, be that hard. Although, you know, another technique that I found is really awesome here, um, when you do have a complex feature that you’re documenting… God, this is, this is sort of sad, the way things have gone, but I’ll tell you, I had a complicated feature, and it was complicated enough that an engineer said, “Let’s meet. I’m going to talk through this.” And they got, she got in there, and she was whiteboarding it, and I was like, “Oh, God, um,” and I said, “Let’s record this, because I know I’m not going to get it all.” So I took the recording and made a transcription – automated – uh, took pictures of the diagram and threw them all into an AI, AI tool, had the AI tool kind of give readable versions of this. The engineer was, like, one of those super smart people that has trouble making it understandable for simpletons like me. So, um, but the AI tool was able to basically take the meeting transcription and the diagrams and get a first pass at the documentation really well. Um, I spent most of my time drawing the diagrams, which was something that I couldn’t automate, and I turned it around in, like, two days, and she gave me a peer bonus for it, which is, like, Google’s little way of giving you little pats on the back with an extra 150 bucks or something. So, uh, I was pretty floored by that, because, like, it was very little lift on my part. Um, I just had to basically kind of shape the AI’s first draft, but it got me, like, 60, 70% of the way there, and I could focus on other things.

So, anyway, um, there are still meetings that I have, but really, like, God, if there’s a feature that’s that confusing, it shouldn’t even be a feature, right? They should, like, make it more intuitive.

“A change to fix an engineering mistake, if it’s significant enough change, if it is, is a significant enough change, perhaps a tech writer could propose a positive way to frame the change.” Yeah, yeah, like, um, there’s definitely ways to throw in a euphemism and be like, “Oh, we decided we found a way to streamline and make this, uh, code more efficient,” and, and instead of saying, “Hey, the engineers, they, they didn’t know what they’re doing, were kind of stupid,” or something, right? You would never say that. But, uh, yeah, you can always spin things. Um, it is good to try to find the positives in things.

“Wow, I might experiment and see if I can use something like that.” Okay, don’t know what that refers to. Hopefully, it was something in the presentation, but maybe it was one of these previous comments. Um, but anyway, thanks, Helen.

“Record the meeting and get AI to draft something.” Yeah, okay. Yeah, and, uh, gosh, you know, the whiteboard thing, don’t underestimate that. Engineers love to draw on the whiteboards, right? And by the time they’re done, it looks like a football play, um, but AI can now process that stuff, and it is dang good, and it’s like, it takes a shot at, at unpacking it in a way that’s super helpful.

Again, I just wanted to mention that, like, or just to emphasize that even though I’ve been very pro-AI in all this, I still really think that, like, for other activities outside of writing release notes and documentation that you don’t really want to be doing anyway, by and large, um, you really, like, if you offload it to AI, you miss out on, like, the whole point of doing an activity. Uh, today I was, I was doing an experiment. I was, I went up to Starbucks, and I’m like, “I’ve heard of this iterative process where, you know, as soon as code can start writing its own code, or an AI can write, start writing its own code, then, um, you know, it gets on this track of iterative improvement and just keeps going up and up.” So what if I could take the… I’ve got this, uh, book review club, or book club, I have at work about, like, automotive and transportation books, and I had all these notes from a book we’d read on, like, the history of the building of the interstate highways (which sounds really boring, but it was actually quite interesting), and I wanted to write a book review of it. But I like experimenting, so I said, uh, “Let me feed this all, all my notes into an AI tool, tell it to write a book review, and then I’ll basically ask it to review what it’s written and look for improvements, and then make another draft, and keep doing this.” And actually, I was using multiple AIs. Be like, “Okay, AI, write this draft,” and then I feed it into another AI and say, “Critique this draft. How can it be improved?” And I take the, the suggestions for improvement, plug it back into the, the authoring AI, and just keep going around and around. Went around 20 times, just after, like, an hour of doing this, and I thought, “My God, this is going to be an amazing, amazing thing,” and, um, uh, it was okay. Like, you know, it’s, like, a book review of the, this book that nobody cares about, this 15-years-old, on the making of the highways. Um, but I was just thinking, left me feeling empty. I was like, “Well, what’s the point of that? Like, I don’t care. Like, I really want to engage in something.” Part of the fun of writing is, like, thinking about things and having your perspective suddenly shift and see things new- newly, uh, in a new way, or just, I don’t know, um, express yourself. And if you have all that, an AI doing all that, it takes away the impetus to write, and I didn’t like that.

So for creative endeavors, I’m losing my, my excitement. I’ll still use it to fix grammar errors, in the same way that I would use Grammarly (which, by the way, now has some nice little AI tools), but, uh, for documentation, I don’t really feel that way. I’m very happy to have an AI tool take a shot at the first draft and get me most of the way there. Um, release, release notes, it’s not something that is kind of fulfilling. The quicker I can get it done, the better.

So, all right. Uh, I don’t know how long we’re supposed, supp- to go. Is this, um, do we have seven more minutes, or, or am I actually… are we running out of time?

Swap: No, what’s our time looking like? We’ve got seven more minutes, because people might want to cut off at that 1 p.m. But yeah, look, this has been fantastic. Like, this, you, we could almost have you present monthly about your insight into, you know, what you’re looking at from a tech writing perspective, and I’m sure there’ll be people queuing up to listen to you. This, this is speaking from the heart, pretty much, so thank you.

Yeah, you’ve got seven more minutes. You’ve still got the floor, so yeah, go ahead.

Well, you know, talking about, uh, meeting more regularly, if you, if you have multiple tech writers in your group, in your, in your, uh, organization at work, wherever you are, consider creating, like, a little study group based around AI or something else. Um, uh, I do this at, at my work, and it’s a lot of fun, because we have writers from all over the org – about between, like, six to eight come each time, even though there are, like, 120, like, members of it – and it’s a way to just kind of, like, um, keep yourself actively engaged in things and to kind of get to know other people, but it’s a lot of fun. Um, let’s see.

“Per Kiny, totally agree about perspective on writing and reasons for doing it yourself. Controversial point of view: I don’t feel that a lot of tech writing is interesting writing and happy to automate my way out of it.” Yeah, yeah. I mean, uh, there’s, there’s kind of, like, a, a time to go slow and enjoy things, right? And another time to go fast. So, um, yeah.

“Luxury is having another technical writer in the organization.” Oh, I’m sorry. Yeah, well, that’s why you’ve got this group, right? We’ve got online groups. Um, I, to be fair, I, uh, I’m sort of a, a lone-ish writer. I have a group of, like, I don’t know, eight or nine other people, but I’m the only one in my automotive organization, so I don’t often work with them, sadly. Really smart people, though.

“We’ve been building a group at my job internally. It has been a fun way to learn and to see how people are feeling while everything’s changing.” Yeah, yeah. Um, there’s never been a topic like AI that, that creates such a hold on the attention of people, uh, in, in our job. I mean, I’ve been… I feel like so many things have changed, and, uh, I would like to get to the point where I stop sort of blogging about AI, but it, it still seems like one of those, one of those massive shifts that’s just not going away yet, anyway.

“In the last person problem in many organizations in Australia, tech writer is a standalone role.” And I, and I think that it will probably continue to go in that trajectory, because if you’ve got AI tools that can expand and amplify what you can do, I think that tech writers will basically just be able to do more with less, uh, cover more ground, not have, like, interns or junior writers that are doing a lot of the tasks that have now become push-button. So, um, yeah, it’s… that’s why I was kind of going off on the tangents on this, the sadness about the increasing isolation. And, um, it… whether that’s just a fleeting sort of thought or if it’s a real trajectory, I don’t really know. But, uh, definitely, if you’re a lone writer, you sure, you feel that, and maybe, maybe have other social avenues to try to collaborate. Um, if you have any other thoughts on this, feel free to reach out. I, I do have, like, a whole series on my blog, uh, like I said, um, where was it? Yeah, right here, “Prompt Engineering for Techcom Scenarios.” I’m going to change “Prompt Engineering” to probably “Generative AI for Tech Writing,” or something, because “Prompt Engineering” doesn’t quite capture all the complexity of the workflows. Like, what I was just describing was, like, an eight-step process for generating release notes, whereas “Prompt Engineering” tends to be like, “Oh, let’s just word this prompt in a, in a fun, specific way and get the result.” It’s a lot more than that.

But, um, definitely experiment with this if you haven’t. Get your hands on the file diffs, throw that in into AI, get the readable summary. It’s, it’s a game-changer. Uh, this is by, this is by far, hands down, the most successful AI technique that I’ve implemented, based… I mean, I’ve done a lot of different things, but this one works, and it’s worked not just for me, it’s worked for other people. Um, it can, it can give you control in a way that I’d never felt, and autonomy and authority, uh, in a way that, that’s fun. Um, so yeah, and it’s just the start. As soon as, uh, more complex workflows become available, um, release notes might get even more, more automated and more AI-driven.

So thanks again for coming to this, and appreciate all of your, um, feedback and thoughts. If you want to grab these slides, you can go to this link here. I’ll just paste that, um, in the little chat, and, uh, as well as the article that’s in there as well. But thanks so much for coming. Feel free to email me or, or shout out on Slack. I’m sometimes there and other channels. So thanks again.

Thank you, Tom. That, that was really good. So, um, thank you again for your time, and like I said before, we’ve recorded the session. I’ll put it up on YouTube in a couple of days, just make sure everything’s nice and clean. But yeah, look, this has been excellent, Tom. I really wish we could do this again, and I hope we get you on the call again, because this, you know, like I said, a lot of people have got into tech writing, have been following your blog for years, and your name comes up far more often than anyone else when it comes to tech writing. So really appreciate that you jumped on this call and shared all these things, like the AI part, but also just speaking from the heart in terms of, you know, where you see, overall, what’s happening with the tech writing industry and how AI is affecting, um, tech writers and stuff like that. So thank you again.

Yeah, thank you. I appreciate, appreciate you having me here.