# Hedge 309: DNS Persist

An episode of [The Hedge](https://invisiblenetwork.ai/podcast/1473947013), 19 Jun 2026, 38 min.

Audio: https://media.blubrry.com/hedge/media.blubrry.com/hedge/content.blubrry.com/hedge/hedge-309.mp3

As DNS is more widely used to distribute certificate information, proving ownership of a resource becomes more critical. The constant challenges required to prove resource ownership, however, increase delay in connecting or using a resource. DNS persists–as the name implies–creates a persistent connection between a resource and a certificate authority. Henry Birge-Lee, Michael Slaughter, and Shiloh Heurich join Russ and Tom to explain how this new record type works and it’s importance to DNS. https://media.blubrry.com/hedge/media.blubrry.com/hedge/content.blubrry.com/hedge/hedge-309.mp3 download

## Transcript

[0:01] - Join us as we gather around the hedge,
[0:03] - where we dig into technology,
[0:05] - business, and culture with the finest minds in
[0:08] - computer networking.
[0:21] - Well, hello, Tom, and hello, Audrey.
[0:23] - Glad to see Audrey's doing well.
[0:25] - Yeah. Yeah. Audrey's good. I'm good too. I
[0:28] - I I missed Audrey earlier this morning. We
[0:31] - were gonna do a recording, and it fell
[0:33] - through for whatever reason.
[0:34] - Changes in time, changes in attitude, changes in
[0:37] - latitude.
[0:39] - Someone lost a flip flop, I think. I
[0:41] - I don't know.
[0:43] - Or something
[0:45] - or a pop top and ended up in
[0:47] - their flip. I don't know. Whatever it was.
[0:49] - So yeah. Alright. I do not have any
[0:52] - Jimmy Buffett on my turntable downstairs. That would
[0:55] - be too hard for me to do work
[0:57] - to.
[0:58] - Van Morrison, I can do, you know, and
[1:00] - Miles Davis, but the Jimmy Buffet's a little
[1:02] - bit
[1:03] - outside my ability to I do have Jimmy
[1:06] - Buffet, but it's upstairs. Anyway, alright.
[1:09] - So let's go over here to Henry. Henry
[1:11] - Virgil, where are you? And just, like, hello
[1:14] - and Yeah. So I'm in San Francisco right
[1:16] - now. I think last time I was on
[1:17] - the podcast, I was a researcher at Princeton.
[1:20] - I since left Princeton and became CEO of
[1:23] - a cybersecurity startup,
[1:24] - and then hence why I'm in San Francisco.
[1:30] - Oh, interesting.
[1:32] - Yeah. I'm not sure if that's, well, anyway,
[1:35] - I don't know.
[1:38] - Far different places, Princeton and San Francisco. Far,
[1:41] - far different places.
[1:42] - There used to be I don't know if
[1:44] - it's still there or not. There used to
[1:45] - be a Mexican place,
[1:47] - there in Princeton
[1:48] - that it was a bring your own bottle
[1:50] - place, whatever. And they have the absolute best
[1:53] - nachos on the face of the earth, Like,
[1:55] - fresh cooked,
[1:56] - you know,
[1:58] - like, they cut them and they cooked them
[1:59] - when you ordered them. They came out of
[2:01] - the oil hot, and, I mean, they were
[2:03] - so, so good. But it's been so many
[2:05] - years since I've been in Princeton that I
[2:07] - don't even know if that place is still
[2:09] - around. Alright. Shiloh, where are you other than
[2:12] - in the woods?
[2:14] - I am. I am quite literally in the
[2:16] - woods. I,
[2:18] - I actually moved from where Henry just moved
[2:20] - to, to Maryland,
[2:22] - and, I live in in a wooded part.
[2:24] - And
[2:25] - I, am at Fastly. And,
[2:28] - there, I was one of the founders of
[2:30] - the certification authority we run called Certainly and,
[2:35] - and do a lot of,
[2:37] - contributions
[2:38] - to Boulder for Let's Encrypt
[2:41] - operations and things. So,
[2:43] - this was very natural piece for,
[2:46] - for me to work on because it benefits,
[2:48] - a lot of people.
[2:49] - Cool. And, Michael, I see a shirt that
[2:52] - looks like a college shirt. Is that a
[2:54] - is that a That is correct. Georgia Tech
[2:56] - yellow jackets represent.
[2:58] - I am. My my father went to Georgia
[3:00] - Tech, so keep going. Excellent. Excellent.
[3:03] - Great taste. I am here in Atlanta, Georgia
[3:06] - as you might expect,
[3:10] - And I am,
[3:12] - well, my day job is with Amazon Trust
[3:13] - Services, which is Amazon's public,
[3:16] - certificate authority.
[3:18] - Okay.
[3:19] - That's cool. Good. Awesome. Alright. So we are
[3:22] - here to talk about
[3:25] - DNS persist, but let's not talk about that
[3:27] - first.
[3:28] - Let's just back into the problem and talk
[3:31] - about certificates. I don't know who wants to
[3:32] - address this.
[3:34] - I know Shiloh had started talking about this
[3:36] - before.
[3:37] - But, like, what is the problem with certificates?
[3:40] - Like, what are we doing with certificates,
[3:42] - and why are we seeing this happen? I
[3:44] - know you said before certificate lifetimes are shortening.
[3:47] - What's going on there? Like, what are we
[3:49] - what what are we facing here?
[3:51] - Well, there are threats for certificates that are
[3:54] - that are in use,
[3:56] - and they there's, cryptographic
[3:58] - threats and,
[3:59] - operational
[4:00] - key exposure threats and things. So tightening up
[4:03] - lifetimes
[4:04] - is a is a security posture improvement,
[4:07] - across across,
[4:09] - the industry. So,
[4:11] - in doing that, that pushes automation to be,
[4:15] - front and center in all all aspects of
[4:17] - keeping,
[4:18] - things online because everything uses TLS, and then
[4:21] - TLS uses x five zero nine and and
[4:23] - that this
[4:24] - is an integral part to the entire Internet
[4:26] - infrastructure.
[4:28] - So,
[4:29] - the the methods that came along with Acme,
[4:31] - which is the standardized way of doing of
[4:33] - doing certificate operations,
[4:36] - only allowed for for,
[4:39] - security
[4:41] - operations that were online and,
[4:44] - done at the time of the certificate issuance.
[4:47] - And that creates a lot of operational friction
[4:49] - and and for different parts of, the industry.
[4:53] - And so we presented,
[4:55] - an alternative, which is based on the DNS.
[4:59] - Okay. So when you shorten certificate lifetimes,
[5:02] - you're not only putting pressure on DNS, you're
[5:05] - actually putting pressure or not on DNS, on
[5:07] - the automation side. You're also actually putting pressure,
[5:11] - like, you have to be Internet connected at
[5:14] - all times to make that that automation work.
[5:17] - You also have,
[5:19] - the other it seems to me like there's
[5:21] - a trade off even on the security side
[5:24] - of having short lived certificates
[5:26] - in that things get desynchronized
[5:30] - and, you know, there are certain patterns that
[5:33] - can emerge from that. Is that a correct
[5:36] - assessment or is, like, am I, like, you
[5:38] - know is shorter certificate life always better or
[5:42] - is it there are So I think there's
[5:43] - a couple of trade offs across the ecosystem.
[5:45] - The first is the,
[5:47] - I think what Shiloh was talking about, the
[5:49] - renewal friction.
[5:50] - So the fact that now a lot of
[5:52] - orgs today still have, like, a person who
[5:54] - manages certificates,
[5:56] - that person makes a change in DNS and,
[5:58] - like, you know, once a year goes and
[5:58] - installs a bunch of certificates, and that type
[5:59] - of flow is just not gonna work with
[6:01] - bunch of certificates, and that type of flow
[6:04] - is just not gonna work with shorter lifespans.
[6:06] - And then the risk, that I think is
[6:08] - very relevant to this group is that you
[6:10] - might just take that, like, master DNS key
[6:13] - that that person used to be using and
[6:14] - spread it out to all of your servers.
[6:16] - So now they can make that DNS change
[6:18] - in an automated fashion.
[6:20] - But now you just put your, like, keys
[6:22] - to your kingdom, your DNS key on every
[6:25] - server. So I think there's sort of some
[6:26] - operational risk with shorter serves
[6:29] - about how people are gonna implement the automation,
[6:31] - and are they gonna risk doing something wrong?
[6:34] - And then I think, you know, we also
[6:35] - work a lot in the broader certificate ecosystem
[6:38] - on the CA side and the certificate transparency
[6:41] - side.
[6:42] - You know, a lot of these CAs are
[6:43] - doing several million certs a day. And
[6:46] - every time you cut lifespan,
[6:48] - that cert count a day
[6:50] - goes up. And then the need for a
[6:52] - CA to be on,
[6:55] - goes up. I think one of the interesting
[6:56] - conversations I had with the Let's Encrypt people,
[6:58] - they said if there's a seven day cert,
[7:01] - that means that, you know, people are gonna
[7:02] - renew after, like, four days of, like, three
[7:04] - days of runway.
[7:05] - And then
[7:07] - if Let's Encrypt were to somehow go out
[7:09] - for three days and their SRT team is,
[7:11] - like, five people that would have to get
[7:13] - on an airplane and fly to the data
[7:14] - center. They couldn't get there in, like, three
[7:16] - days, half the Internet would go down. So
[7:19] - you put more strain on sort of, you
[7:21] - know, CAs to be boys up. And then
[7:23] - also the certificate transparency infrastructure,
[7:26] - people talk about a lot now. They log
[7:28] - every cert that's signed, and then the signing
[7:30] - volume goes up. So there's a lot in
[7:31] - the ecosystem changing to support this,
[7:34] - but we're really looking at the sort of
[7:36] - operational friction.
[7:40] - Okay. That's kinda cool. And so yeah. So
[7:43] - there are also there are real human cost
[7:45] - to this, not just automation. Right? If your
[7:49] - if your cert goes faster,
[7:50] - then that becomes
[7:52] - essentially
[7:53] - a,
[7:55] - not really a an attack surface, but a
[7:57] - failure surface
[7:59] - in the network
[8:00] - that become or in the system, not necessarily
[8:02] - just the network, but in the system that
[8:04] - says, okay, well, if this goes this goes
[8:06] - out, oh my, now I have a problem
[8:09] - that I have to go solve. And like
[8:10] - you said, people may have to get on
[8:11] - an airplane and fly. Well, that, like, that
[8:14] - really stinks
[8:15] - because that's, you know, family and
[8:18] - I don't know, fuel and all the other
[8:20] - stuff that goes with that. That's that's really
[8:22] - painful.
[8:23] - Yeah. Alright.
[8:24] - So all of that said, anything else you
[8:27] - wanna bring up there, Tom, before we jump
[8:29] - into?
[8:30] - No. I have a couple of questions for
[8:31] - later on. Okay. Cool. Alright. So
[8:36] - all that said,
[8:38] - if we what the solution that you're talking
[8:41] - about let's talk about so that's kind of
[8:42] - the problem.
[8:44] - So how are you thinking about structurally, like,
[8:47] - architecturally?
[8:48] - How are you thinking about
[8:51] - solving this? You know, not without getting not
[8:53] - no technical details at this point. Just like,
[8:56] - what is the general pattern? What are you
[8:58] - trying to to do to solve this?
[9:01] - Well, I think because of, you know, some
[9:03] - so Henry did security analyses,
[9:05] - and Michael put, profiles in place at the
[9:08] - CAB forum that allowed for the idea that
[9:11] - you could replace this sort of online,
[9:15] - challenges
[9:16] - with a standing authorization
[9:18] - that says,
[9:19] - I authorize
[9:21] - this Acme account,
[9:23] - to always make changes on the to you
[9:26] - know, in certificates by issuing new certificates,
[9:29] - on this on this name, on this, FQDN.
[9:35] - Okay.
[9:36] - So anybody want to expand on that a
[9:39] - little? Go ahead. Sorry. Yeah. I think it's
[9:41] - it's true that the thing that certificates are
[9:43] - best at is expiring, so they're time bombs.
[9:46] - And so
[9:48] - this is not effectively a new problem.
[9:50] - For years, ever since Acme was started, there
[9:53] - was DNS based domain control verification,
[9:56] - and there were smart people that, meanwhile, that
[9:59] - have figured out ways to
[10:01] - work around the limitations and reduce the risk
[10:04] - of,
[10:05] - having
[10:06] - a outage related to a certificate expiring because
[10:09] - you were unable to renew the certificate
[10:12] - because
[10:13] - of reasons.
[10:14] - And a lot of those mechanisms,
[10:19] - essentially equated to something that the term Henry
[10:21] - used. So I'm leveraging it from him, which
[10:24] - is CNAME magic,
[10:25] - in which you would
[10:27] - do something similar in which you'd add a
[10:29] - CNAME that delegates
[10:31] - that domain
[10:32] - or that, basically, that renewal or that domain
[10:35] - control validation action to another entity.
[10:38] - Most of the time, it is a CDN
[10:40] - or some other infrastructure provider. Sometimes it's a
[10:43] - single entity within an organization or some enterprise
[10:46] - use cases.
[10:47] - But it's another way of reasoning about this
[10:50] - problem of how do I prevent these clients
[10:53] - individually having to
[10:55] - each being exposed to this risk of being
[10:57] - unable to renew successfully.
[10:58] - How do I centralize in that particular model,
[11:01] - solve it in one place, and do it
[11:03] - in a way that makes sense?
[11:05] - And
[11:06] - this builds upon that legacy about how do
[11:08] - we do that in a more straightforward and,
[11:11] - direct way and explicit way than what folks
[11:14] - were doing on their own. So are you
[11:16] - doing a c name on the CAA?
[11:19] - Is that basically
[11:21] - is is that I'm trying to understand in
[11:23] - my head, like, how this
[11:25] - If you wanna get into the mechanics, happy
[11:27] - to. Would love to. Well, well, I don't
[11:29] - know if if anybody else wants to describe
[11:31] - more of the,
[11:32] - more of the overall problem and solution before
[11:34] - we jump there, if Tom has questions. But,
[11:37] - yeah.
[11:38] - So, basically, you are delegating
[11:42] - certificate
[11:43] - renewal
[11:45] - to a third party.
[11:47] - Now is this for on this is for
[11:49] - ongoing operations or for in times of emergency?
[11:52] - Right? It's both players Or or is it?
[11:55] - I think This used
[11:58] - Yeah. I'm sorry. I think we have a
[11:59] - Go ahead.
[12:01] - Collision lag.
[12:02] - I was gonna say, just to be clear,
[12:04] - this is for peep this is largely what
[12:06] - people used to be doing. They would use
[12:08] - a c name to delegate certificate renewal
[12:11] - to a third party Oh, okay. On an
[12:13] - ongoing basis. And I will foreshadow a little
[12:16] - bit what we're trying to do. You know,
[12:17] - a c name is a particular vehicle
[12:20] - in DNS,
[12:21] - and this was not really the the number
[12:24] - one rule of the c name. And you
[12:25] - can also imagine once you delegate with that
[12:27] - c name, that third party has to run
[12:29] - this piece of infrastructure
[12:31] - that essentially is the target of that c
[12:33] - name and constantly changes a bunch of DNS
[12:36] - records just to get certificates.
[12:39] - So you move the failure point outside of
[12:42] - the primary zone to this secondary c name
[12:44] - delegated zone, but you still have this, like,
[12:47] - always up infrastructure that can go down, can
[12:50] - break, introduces complexity.
[12:53] - Yeah.
[12:54] - And and interestingly,
[12:55] - c names are imperfect even in this regard
[12:58] - because
[12:59] - c names are cached and flattened whenever possible
[13:03] - by recursive,
[13:05] - resolvers.
[13:05] - And so you you you flatten the c
[13:08] - name today at 10:00,
[13:10] - and then somebody changed the delegation or whatever.
[13:13] - And now you're basically out until that cache
[13:16] - runs out, and that recursive resolver says, oh,
[13:20] - my cache ran out. My TTL expired. I
[13:23] - need to go reflatten that c name. And
[13:25] - so that's gonna cause lots of angst and
[13:27] - problems
[13:29] - in that process. Just that just that TTL
[13:32] - and etcetera that goes into this. Yeah. So
[13:35] - that's I can see where that's an issue.
[13:38] - The, the problem with CNAME is that you
[13:41] - cannot have more than one.
[13:43] - And so that Okay. That's huge problem for
[13:46] - multi CDN, multi CA
[13:49] - customers.
[13:50] - Yeah. Right? And that's where you want to
[13:52] - authorize
[13:53] - to,
[13:54] - things to to do this on your behalf.
[13:57] - And yeah.
[13:58] - So I so I know d names are
[14:00] - new, and a lot of people don't like
[14:02] - them. Would d names solve part of that
[14:04] - problem?
[14:05] - Yes. Oh, oh, boy. I'm getting a lot
[14:07] - of
[14:09] - don't go there. I I don't think a
[14:11] - d name is fully the answer. I think
[14:13] - of the d name more as helping with
[14:15] - sort of sub labels, you know, below the
[14:17] - c name.
[14:19] - But I I think it's it's a good
[14:20] - point to transition into the solution, which is
[14:23] - just not to use not to use c
[14:25] - names anymore, at least not for this type
[14:27] - of online renewal stuff. Okay.
[14:31] - Okay. Yeah. That sounds good. So so yeah.
[14:34] - So if you're not using c names,
[14:37] - how what, like, system are you using? Are
[14:40] - you creating a new
[14:42] - record type?
[14:44] - EXT.
[14:46] - Okay.
[14:48] - We yeah. And we are, you know,
[14:51] - fundamentally, you're just trying to prove control of
[14:54] - a Danus
[14:55] - name
[14:56] - in in Acme, right, and in DCV for
[14:58] - DCV. And
[15:00] - you
[15:01] - in the previous model, you would look at
[15:03] - that,
[15:04] - DNS name as an a record and talk
[15:07] - to port 80 on the thing or or,
[15:10] - and and now or trail the or trace
[15:13] - the c name. Right? And, now what you
[15:16] - can do is you can have TXT records
[15:18] - at more of course, more than one at
[15:20] - that same exact, DNS
[15:23] - record that says that
[15:25] - this CA is authorized
[15:27] - and this account at that CA is authorized
[15:29] - or this other entity at that a subscriber
[15:32] - of some sort at that CA is authorized
[15:34] - to issue certificates
[15:36] - to this name, to this DNS name that
[15:38] - I'm part of. And, of course, you have
[15:41] - you have the the DNSSEC, and you have
[15:43] - things that are that are that are sealing
[15:45] - that, part of it. So that's your,
[15:49] - you you're proving now what you wanted to
[15:51] - prove, which is control of that domain name.
[15:54] - Okay. Interesting.
[15:55] - So I guess my I guess my when
[15:58] - whenever when whenever anyone says TXT,
[16:01] - my immediate response is gonna be the same
[16:03] - thing my response is with,
[16:06] - AFI SAFE's in BGP, which I wish we
[16:09] - would not use so many
[16:10] - of. Please, there are other ways of carrying
[16:13] - data in in BGP than AFI SAFE's. And
[16:16] - we just keep like, oh, it's an opaque
[16:18] - LSA. Let's just throw more stuff in there.
[16:21] - So I guess my my initial reaction is
[16:24] - always gonna
[16:25] - be why TXT? Why not a new record
[16:27] - type?
[16:28] - Are we perhaps causing confusion in the TXT
[16:32] - space
[16:33] - by doing this? And I'm not saying it's
[16:34] - invalid, I'm just saying those are like initial
[16:36] - reactions that always come to mind whenever anyone
[16:40] - starts using a TXT or an AFI SAFI
[16:43] - or whatever it is. An opaque LSA in
[16:45] - in OSPF.
[16:46] - Is there,
[16:48] - I mean, are those
[16:50] - other things were considered, but TXT ended up
[16:53] - being best? It was the easiest path forward.
[16:57] - I don't know. I'm just asking, like, how
[16:58] - does that how does that work? Largely on
[17:00] - following precedent.
[17:02] - So the primary way of do validating domain
[17:06] - control is a TXT record at _acmechallenge,
[17:10] - .dom yourdomain.com.
[17:13] - And then the IETF actually, you know, came
[17:15] - out to endorse this eventually. They have, like,
[17:17] - a best practices
[17:19] - recommendation,
[17:20] - like RFC on domain control validation, which says
[17:23] - the best way to do it is to
[17:24] - show
[17:25] - uploading of a TXT record
[17:27] - at, you know, that underscore prefix label. And
[17:30] - so we had you know, that's what the
[17:32] - ecosystem is used to, and that was what
[17:34] - used to be behind those c name records
[17:37] - we were talking about. So you used to
[17:39] - follow the c name to an ephemeral TXT
[17:41] - record.
[17:42] - And, you know, kind of keeping with the
[17:44] - consistency
[17:45] - just to let's use the same as similar
[17:47] - looking of a record as possible. Yeah. But
[17:50] - just put it where the c name used
[17:51] - to live and just make it a persistent
[17:53] - delegation to, like, say, hey. That can sign
[17:55] - my account
[17:56] - again and again.
[18:00] - Okay. And that makes sense because people have
[18:03] - already written code. People have already written runbooks.
[18:06] - They know how to do this. You're just
[18:08] - kind of changing what they're already doing
[18:11] - rather than inventing something absolutely totally new
[18:14] - in in the environment, which then makes sense.
[18:17] - Yeah.
[18:17] - Okay. So that's perfectly valid answer. I just
[18:20] - always ask. Right? Because I I do see
[18:23] - people do stuff and I'm like,
[18:25] - you know, yes. It's opaque. That doesn't mean
[18:27] - you can throw anything you want to at
[18:29] - it.
[18:30] - Yeah.
[18:31] - Which we do. We do a lot.
[18:35] - Even I think you raise a great point
[18:37] - about following precedent and about people already used
[18:39] - to this. Even the mechanisms on how you
[18:41] - identify
[18:42] - who which accounts or which entity is authorized,
[18:45] - how you identify the CA, they also follow
[18:48] - precedents. We're using a lot of the same
[18:49] - syntax
[18:50] - that the certificate
[18:52] - authority authorization,
[18:53] - CA records utilize
[18:55] - to specify,
[18:57] - the certificate authorities.
[18:59] - The account URI is following,
[19:02] - the ACME structure is allowed as well, and
[19:04] - we go into more details about how that
[19:06] - can be expressed, which is another interesting part
[19:08] - of the conversation.
[19:09] - But so a lot of
[19:11] - the origination of the this direction
[19:14] - is based on what is it that how
[19:17] - does domain control validation already work, and how
[19:19] - can we streamline it and improve it in
[19:21] - a way that has some nice properties but
[19:23] - doesn't
[19:25] - shock anybody about what is going on. Okay.
[19:27] - That that is indeed one of the considerations
[19:29] - that led us here. Okay. And and just
[19:31] - to be clear for people who are listening,
[19:34] - you're not
[19:35] - actually
[19:36] - extending the lifetime of a certificate. You're you're
[19:39] - actually saying
[19:40] - these short life certificates or shorter life certificates
[19:44] - are actually better in a security from a
[19:46] - security perspective.
[19:48] - We're just gonna find a way to make
[19:50] - it easier to automate and manage
[19:53] - those shorter lifetime certificates.
[19:55] - So yeah. Just okay. Good.
[19:58] - That is correct. And one of the interesting
[20:00] - parts of the way that public certificate,
[20:03] - the public e public e PKI works, excuse
[20:06] - me, is that there are actually two timelines
[20:08] - that are being managed
[20:09] - behind the scenes. There's the when you perform
[20:11] - a domain control validation,
[20:13] - that is a femoral action that get at
[20:16] - a point in time determination that says, at
[20:18] - this point, I validated that this
[20:21] - applicant is what we call the person requesting
[20:23] - a certificate,
[20:24] - has control over this domain,
[20:26] - and then you can issue a certificate based
[20:28] - off of that
[20:29] - assertion of control.
[20:31] - And so this decouples the ability to issue
[20:34] - certificates or renew certificates
[20:37] - from the ability to,
[20:40] - prove that you have control. Because now you
[20:42] - have a mechanism to have a persistent
[20:44] - means of asserting control, validating that the TXD
[20:48] - record that we're talking about is still in
[20:49] - place, And then your ability to renew certificates
[20:53] - based off of that
[20:54] - existing
[20:55] - domain validation record
[20:57] - is decoupled. So you can issue as many
[20:58] - certificates on whatever
[21:00] - short whatever lifetime you need
[21:02] - with only having to take action on the
[21:04] - certificate rather than having to rotate the renewal
[21:07] - knots, the DNS knots
[21:09] - at the same time. So what else what
[21:11] - else in the ecosystem has to change to
[21:14] - take advantage of this? You published a c
[21:16] - name or sorry, not a c name, a
[21:17] - TXT.
[21:19] - And and so what what other code changes,
[21:21] - what practices change to make this work?
[21:26] - Well, the the policy change in the cap
[21:28] - form is already in place. So what's,
[21:31] - what needs to happen is the implementation of
[21:33] - this, and,
[21:35] - that's,
[21:36] - going on this year now already with, Boulder,
[21:39] - which is for let's encrypts back end, and,
[21:42] - it's going on with, a
[21:45] - number of client implementations as well that are
[21:47] - used on, at least, on large scale,
[21:50] - hosting providers.
[21:53] - So so there's software changes, right, in reading
[21:55] - the record correctly? Because it used to be
[21:57] - a c name pointing at a text, and
[21:59] - now it's a text.
[22:01] - Alright. So now
[22:02] - I have that piece so that so you're
[22:04] - saying the things like bind or whatever other
[22:07] - tools that you use
[22:09] - are actually being built now to handle this.
[22:12] - So that if I'm an end user,
[22:14] - if I download the latest update or whatever,
[22:17] - those things are built in.
[22:19] - And I think
[22:20] - Well, bind wouldn't need to be updated, would
[22:22] - it? Yeah. Not bind, but whatever. That's nice.
[22:24] - Because it's t f c. It's t x
[22:26] - t r t. Exactly.
[22:28] - Yeah.
[22:29] - The the yeah. The, the providers, like Let's
[22:32] - Encrypt,
[22:33] - and certainly will be providing it, like, this
[22:36] - year. And then so as soon as clients
[22:39] - update, they can start taking advantage of this.
[22:41] - And
[22:42] - it,
[22:43] - it should just make their friction go down.
[22:47] - This will manifest as another
[22:49] - domain control validation method
[22:51] - that is an option for,
[22:54] - clients to select and servers to support. So
[22:56] - there has to be support for this method
[22:58] - on both sides, the client and the server
[23:00] - side.
[23:02] - And when you say client, are peep should
[23:04] - be people be thinking of web browsers, stuff
[23:06] - like that? Or should they be thinking what
[23:09] - yeah. What should they be thinking of? Yeah.
[23:11] - Acme clients. So more like Acme clients. They're
[23:13] - thinking about. Yeah. Yeah. I kind of assumed
[23:15] - that's what you meant, but I just wanna
[23:17] - make sure, you know, people listening to this
[23:19] - again are probably thinking Well, the client to
[23:21] - a certificate
[23:22] - authority
[23:23] - Right. Is the server to an end user.
[23:26] - Yeah. Yeah.
[23:28] - Right. Right. Yes. Okay. That makes sense. Right.
[23:31] - And so So cert bot is is a
[23:33] - pretty popular one from Let's Encrypt, and that's
[23:35] - that's getting updated in tandem.
[23:38] - Okay.
[23:39] - Awesome.
[23:40] - So we're gonna have these shorter list certificates.
[23:43] - Operationally,
[23:44] - if you run not if you run if
[23:46] - you run a CA
[23:47] - or if you have a certificate in your
[23:50] - organization,
[23:51] - you need to watch for this. Right? And
[23:53] - you need to make sure that is there
[23:55] - anything administratively
[23:57] - that say,
[23:58] - I am
[23:59] - big bank number 48
[24:01] - or whatever my company is, my organization is.
[24:04] - What do I need to do
[24:07] - in relation to this,
[24:10] - other than just update the software?
[24:12] - Is it primarily gonna be on my CA
[24:14] - side?
[24:15] - Like, if I'm I think there's definitely or
[24:17] - not. Several recommendations we would make for this,
[24:20] - you know, hypothetical
[24:21] - big bank.
[24:22] - The first is to try to eliminate
[24:25] - all manual certificate renewal processes
[24:28] - that will always cause increased friction. I know
[24:30] - many enterprises, as I said, are still doing
[24:33] - this manual process where they have, like, a
[24:35] - compliance person who gets the certs from the
[24:37] - CA
[24:38] - and puts them on the servers,
[24:39] - and it needs to go to be an
[24:41] - Acme client that can get certs from the
[24:44] - CA themselves automatically
[24:46] - and then deploy
[24:47] - automatically. What's really good and why we're recommending
[24:50] - people adopt this method today
[24:53] - is that the prior generation of Acme clients
[24:55] - needed to have some type of access to
[24:58] - either the HTTP directories on those servers
[25:01] - or to the DNS infrastructure. And what we're
[25:03] - saying is your Acme client doesn't need to
[25:05] - do any of that. But you can still
[25:07] - have that compliance person go in, write this
[25:09] - DNS record, put it in DNS,
[25:12] - and we've decoupled that. So one thing we're
[25:15] - doing is actually making it easier for enterprises
[25:18] - to lean into that adoption,
[25:21] - get the ACME plans out there, get the
[25:22] - automated certificate management, and still have, you know,
[25:26] - secure DNS zones that don't have any credentials
[25:29] - anywhere and just put these static records in
[25:31] - place to opt optimize them. So I think
[25:33] - for enterprise that are looking to adopt certificate
[25:37] - automation,
[25:38] - I would say that this DNS persist proposal
[25:40] - is the strongest,
[25:42] - most recommended way forward right now.
[25:46] - Okay.
[25:49] - And if I were
[25:51] - a provider, I'm gonna be pretty much in
[25:53] - the same position, like a transit or something
[25:55] - like that. Because I just I have customers
[25:57] - and I have CA servers
[25:59] - that I can use Acme client on to
[26:01] - do all the stuff there. And if I
[26:03] - were an IX, it's gonna be pretty much
[26:05] - the same. So really, the place this
[26:08] - it has the most impact is just in
[26:10] - building the TXT records
[26:12] - and
[26:13] - in for certificate authorities, like Let's Encrypt
[26:17] - to manage their end of the of the
[26:20] - situation.
[26:21] - Right? And software developers who are building
[26:24] - the tools.
[26:25] - Is that where you would say the most
[26:27] - impact is? Impact, you mean, like, input into
[26:29] - the ecosystem and the need to change software?
[26:32] - Yeah. Yeah. Right. Right. You actually have to
[26:34] - do something. Yes. Yeah. I these enterprises banks
[26:37] - don't have to.
[26:40] - Okay. Yeah. I'm just thinking through all the
[26:43] - different roles that people might have and, like,
[26:45] - you know, where would would the impact be
[26:48] - different for any of these? But it doesn't
[26:49] - seem like it would be. It's mostly gonna
[26:52] - be on certificate authorities.
[26:54] - Stuff like that is where that end of
[26:56] - it's gonna be.
[26:57] - I'm I'm curious.
[26:59] - Do any of y'all have a sense of
[27:01] - the distribution of methods of domain control authorization
[27:05] - that are used today? Is it, like, mostly
[27:08] - the c name method, or is it balanced,
[27:10] - or is it kind of all over the
[27:12] - place? Or maybe that's hard to get. You
[27:14] - probably would
[27:15] - I guess you'd have to get that from
[27:17] - the CA, but, like, any any sense there?
[27:20] - I have a so we've done research on
[27:22] - that back when I was at Princeton. We
[27:24] - collected Let's Encrypt blogs that included validation data
[27:27] - and then published summary statistics on what validation
[27:29] - looks like.
[27:37] - Let's encrypt boat, we were looking at, like,
[27:38] - 70%
[27:40] - people changing
[27:41] - dot well known
[27:43] - directories on their HTTP servers, and that's one
[27:47] - way to do verification.
[27:48] - And then 30%
[27:50] - with this type of DNS stuff, I don't
[27:53] - think we broke down the DNS to what
[27:55] - I call magic CNAME, which is where you
[27:57] - outsource it to someone else versus in zone,
[27:59] - like, changes.
[28:00] - I would suspect that a lot of the
[28:02] - DNS was being done through the the CNAME
[28:04] - stuff. But I should note that, like, when
[28:06] - you follow electric encrypt instructions, you kinda got,
[28:09] - like, this choice. It's normally, like, a little
[28:11] - cert bot that runs close to the server
[28:13] - dir.
[28:14] - If you go to other CAs, they have
[28:16] - much higher rates of
[28:18] - DNS validation. I don't you know, that logic
[28:20] - is, I think, less public, but I will
[28:22] - say it varies from CA to CA.
[28:25] - So but I I would definitely think north
[28:27] - of 30%. I think Let's Encrypt actually has
[28:28] - a low
[28:30] - percentage of people doing DNS
[28:32] - validation, and it's more popular enterprises too.
[28:38] - Yeah. Well, Let's Encrypt tends
[28:41] - to pull in the people who,
[28:43] - for whatever reason, are not paying
[28:46] - ACA
[28:46] - to do the work.
[28:48] - You know, it's more of an open source
[28:50] - kind of a community based.
[28:52] - And therefore, you're going to get the smaller
[28:55] - companies, the, you know, companies without somebody
[28:59] - on board that's going to do the work
[29:01] - that you're talking about full time. And so
[29:04] - therefore, they're probably just gonna follow, you know,
[29:07] - simplest path forward.
[29:08] - What's the easiest way for me to get
[29:10] - to where I wanna go?
[29:11] - Because I just want my website up in
[29:13] - the web. I just want it running, and
[29:15] - I don't want customers calling me because they
[29:17] - don't have a green lock
[29:19] - on their browser. Thank you.
[29:21] - That's what that's what a lot of people
[29:23] - want. It's, you know, keep fighting.
[29:26] - I think for the most part, people will
[29:28] - just need to update their software as it's
[29:31] - released, and
[29:32] - they will be able to keep going forward
[29:35] - the same way or they and they should
[29:37] - reevaluate
[29:38] - their their needs as far as access goes
[29:40] - and and maybe close down some channels to
[29:43] - create a more secure environment
[29:45] - that they had already.
[29:48] - Okay. So that's all sounds cool to me.
[29:50] - I mean, it all sounds very simple
[29:52] - and
[29:53] - very straightforward, and what you're doing makes sense.
[29:56] - Now do you have any challenges
[29:58] - in
[29:59] - deployment?
[30:00] - Like, do you see any place where you're
[30:02] - thinking
[30:03] - now I know. You know, it's kinda but,
[30:05] - anyway,
[30:06] - like, where we're gonna hit I p v
[30:07] - six moments where people are gonna be like,
[30:10] - no.
[30:12] - Or or do you think it's fairly straightforward
[30:15] - and, you know, it's just gonna roll out
[30:17] - and you'll see
[30:18] - an increasing deployment over time. I wouldn't call
[30:21] - it a challenge exactly, but I think, an
[30:23] - emerging area on the deployment front is just
[30:26] - the existence of
[30:27] - non Acme CAs
[30:29] - and Acme accounts that live under,
[30:32] - non Acme accounts. So maybe this is just
[30:34] - a a brief,
[30:35] - background on the CA industry.
[30:37] - But, you know, some CAs use the automated
[30:40] - certificate management environment protocol. It's a client to
[30:44] - CA protocol
[30:45] - that gets you your certs automatically.
[30:47] - Some CAs have other automation channels,
[30:50] - and then some CAs, you know, don't have
[30:52] - any automation options. But, essentially, Acme represents some
[30:57] - fraction of the ecosystem.
[30:58] - And one thing that we actually have CA
[31:00] - operators talk about is they might have
[31:03] - some Acme accounts that live under sort of
[31:05] - a more global umbrella of, like, an enterprise
[31:08] - account.
[31:09] - So one thing that I think we've put
[31:11] - into the draft more recently
[31:14] - is kind of the ability to do authorization
[31:16] - at the level of this enterprise account and
[31:18] - then have your Acme accounts, like, live underneath
[31:21] - it. So I'd say as we're kind of
[31:23] - interacting more with the broader space
[31:26] - of CAs and users, we're discovering more of
[31:28] - these use cases about, like,
[31:31] - where is Acme? Like, you know, the the
[31:33] - standard IETF model is, like, that is the
[31:35] - subscriber account, the Acme account. But increasingly, we
[31:39] - actually see, like, The subscriber has some type
[31:42] - of external billing relationship with the CA, and
[31:44] - Acme is just the protocol they run on
[31:47] - their server. So I'd say that's a neat
[31:49] - emerging use case, like,
[31:51] - Acme as a as a protocol
[31:53] - and less as, like, your primary account,
[31:56] - you know, relationship with the
[31:58] - CA.
[32:01] - Couldn't they use the same systems? They just
[32:03] - have to have them built into their into
[32:05] - their ecosystem
[32:07] - in the same way, I would think. Yeah.
[32:09] - They do. They use similar things, but
[32:11] - you have this identity that's so because we
[32:14] - authorize on that account identity, you have this
[32:16] - introduction of this other identity
[32:18] - okay. That's kind of the CA account. And
[32:20] - then you have to sort of create a
[32:22] - policy around what does that mean to have
[32:24] - a broader account and map that back to
[32:26] - individual Acme accounts.
[32:30] - Yeah. Alright. Yeah. That makes sense.
[32:33] - So what does it look like? Are you
[32:35] - seeing deployments? Are you seeing lots of deployment?
[32:40] - Are you, like, is it rolling out pretty
[32:42] - fast? Or is it slow? Or what what
[32:44] - do you think the future holds here? Where
[32:46] - where's the where's it looking?
[32:49] - I think on a lot of the major,
[32:51] - open source projects that are, in heavy use,
[32:55] - there there's all active work to to get
[32:58] - integration, and there's there's interest,
[33:00] - across the board because it solves
[33:03] - a lot of operational friction for people,
[33:05] - and,
[33:06] - they they would like to use it.
[33:09] - Okay.
[33:10] - Cool.
[33:12] - Okay. Any
[33:13] - is this a draft? Is this in draft
[33:15] - state? Is it a working group item?
[33:17] - Where is this in the IETF process?
[33:22] - We're between o one and o two of
[33:24] - the draft right now. Work draft. It's a
[33:26] - it's adopted at the We think it's ACME
[33:28] - working group.
[33:32] - Okay. So it's adopted.
[33:33] - It's just okay. Cool. I'm just always curious,
[33:36] - like, you know, what if you're at that
[33:38] - point, the likelihood of major changes coming down
[33:41] - the road
[33:42] - because somebody comes in sideways and says, oh,
[33:45] - no. You should have done it this way
[33:47] - or whatever. Some of the security people or
[33:49] - something. Right? Are much lower once you once
[33:52] - you're adopted into a working group draft and
[33:54] - you're kind of rolling along through the process.
[33:57] - The odds of massive changes are are much
[33:59] - lower.
[34:01] - Something, you know, people when they read drafts
[34:03] - in the ITF, they don't realize that if
[34:04] - it's
[34:05] - certain stages, things are more more likely to
[34:08] - be changed.
[34:09] - And then in other stages and so, you
[34:11] - know, it's a good thing to know that.
[34:13] - Alright. Cool. I don't really have any other
[34:16] - particular questions. It all sounds like a cool
[34:18] - technology,
[34:19] - cool cool solution to
[34:21] - a problem that is approaching.
[34:24] - And, you know, instead of people just typing
[34:27] - faster
[34:28] - and getting on airplanes to carry USB keys
[34:31] - around the servers,
[34:34] - you can actually
[34:36] - build a system where it can be rationally
[34:39] - automated.
[34:40] - So, yeah, that's always a good thing. Yeah.
[34:43] - Sneaker nets never never fun. So I don't
[34:46] - know.
[34:47] - Anything else anybody wants to say before we
[34:49] - wrap up as far as future, current, anything
[34:52] - else?
[34:53] - Henry, anything you
[34:55] - I'm really good. Just,
[34:56] - I think Shiloh is quite quite about the
[34:58] - deployment, but, you know, let's incur test stated
[35:00] - intent to support this. And
[35:03] - Shiloh here is from, certainly from Fastly,
[35:06] - So and, Leica Slaughters from Amazon. Like, we
[35:09] - do definitely see the major CAs at least
[35:12] - coming coming out with this. It's already coming
[35:14] - out in the ACME clients. So this is
[35:16] - gonna be a real option for enterprises to
[35:19] - reduce their certificate friction.
[35:22] - Alright. Awesome. Shiloh, anything from your end?
[35:27] - Yeah. All the major cloud providers, I think,
[35:29] - are interested, and,
[35:31] - it's it's just gonna make things better in
[35:34] - the TLS
[35:35] - ecosystem, which allows better security posture,
[35:38] - overall.
[35:39] - Okay. Anything else, Michael? You wanna cover? Yeah.
[35:42] - Change is coming, and we highly encourage automation.
[35:45] - We don't like seeing certs expire. This is
[35:47] - one mechanism, one additional tool in your tool
[35:50] - belt to prevent
[35:51] - cert related outages.
[35:52] - So we encourage you to use it. Okay.
[35:55] - Cool. Join us on the mailing list for
[35:57] - our ETF if you have comments or concerns
[35:59] - or
[36:00] - Okay. Be happy to take in collaborators as
[36:02] - well. Alright. Tom, any other questions?
[36:05] - No. No. It's been great. Alright. Cool. Alright.
[36:08] - Henry, where can people find you? Do you
[36:09] - have a blog, or are you on Yes.
[36:12] - Some kind of
[36:14] - social antisocial media? Yes. I was LinkedIn, but
[36:16] - I'm quite mad about updating that. I now
[36:20] - blog through our security company, Crosslayer Labs. So
[36:23] - Crosslayer
[36:23] - Labs, all1word,.com,
[36:26] - /blog,
[36:27] - and I have some articles there. Okay.
[36:31] - Awesome. Very good. And, Shiloh, any place you
[36:34] - you are publicly
[36:36] - out there saying things to people or just
[36:39] - living in a way? I think the closest
[36:41] - I get to social media is GitHub,
[36:43] - s hirick on there.
[36:46] - And I,
[36:47] - I do blog occasionally for as part of
[36:49] - Fastly.
[36:51] - Okay. So Okay. Cool. Michael, anything from you?
[36:54] - LinkedIn.
[36:55] - That's it. Okay.
[36:58] - He took your, he took your thing, Tom.
[37:00] - You can't say anything now.
[37:05] - That's alright. Less is more.
[37:07] - Less is more.
[37:09] - Alright. Go ahead, Tom.
[37:11] - LinkedIn.
[37:12] - See. It's it's like a thing. Alright.
[37:16] - I'm Russ White. You can find me here
[37:17] - at the hedge at rule eleven dot tech.
[37:20] - I'm on LinkedIn occasionally,
[37:22] - on exit routing geek occasionally, not very often,
[37:24] - but sometimes I am. And
[37:27] - we've we know we live in an attention
[37:29] - driven economy. Thank you very much for listening
[37:31] - all the way to the bitter end of
[37:32] - this episode of The Hedge, and we will
[37:34] - catch you next time.

Shows are listed from Apple Podcasts' charts and directory, and Podcast Index where it is used. Every show and episode belongs to its publisher and plays from the publisher's own feed; this site hosts none of them.