rfc8963v3.txt		rfc8963.txt
	Independent Submission C. Huitema		Independent Submission C. Huitema
	Request for Comments: 8963 Private Octopus Inc.		Request for Comments: 8963 Private Octopus Inc.
	Category: Informational December 2020		Category: Informational January 2021
	ISSN: 2070-1721		ISSN: 2070-1721
	Evaluation of a Sample of RFCs Produced in 2018		Evaluation of a Sample of RFCs Produced in 2018
	Abstract		Abstract
	This document presents the author's effort to understand the delays		This document presents the author's effort to understand the delays
	involved in publishing an idea in the IETF or through the Independent		involved in publishing an idea in the IETF or through the Independent
	Stream, from the first individual draft to the publication of the		Stream, from the first individual draft to the publication of the
	RFC. We analyze a set of randomly chosen RFCs approved in 2018,		RFC. We analyze a set of randomly chosen RFCs approved in 2018,
	skipping to change at line 47 ¶		skipping to change at line 47 ¶
	implementation or deployment. Documents approved for publication by		implementation or deployment. Documents approved for publication by
	the RFC Editor are not candidates for any level of Internet Standard;		the RFC Editor are not candidates for any level of Internet Standard;
	see Section 2 of RFC 7841.		see Section 2 of RFC 7841.
	Information about the current status of this document, any errata,		Information about the current status of this document, any errata,
	and how to provide feedback on it may be obtained at		and how to provide feedback on it may be obtained at
	https://www.rfc-editor.org/info/rfc8963.		https://www.rfc-editor.org/info/rfc8963.
	Copyright Notice		Copyright Notice
	Copyright (c) 2020 IETF Trust and the persons identified as the		Copyright (c) 2021 IETF Trust and the persons identified as the
	document authors. All rights reserved.		document authors. All rights reserved.
	This document is subject to BCP 78 and the IETF Trust's Legal		This document is subject to BCP 78 and the IETF Trust's Legal
	Provisions Relating to IETF Documents		Provisions Relating to IETF Documents
	(https://trustee.ietf.org/license-info) in effect on the date of		(https://trustee.ietf.org/license-info) in effect on the date of
	publication of this document. Please review these documents		publication of this document. Please review these documents
	carefully, as they describe your rights and restrictions with respect		carefully, as they describe your rights and restrictions with respect
	to this document.		to this document.
	Table of Contents		Table of Contents
	skipping to change at line 1108 ¶		skipping to change at line 1108 ¶
	+------------+------------+-------+---------+		+------------+------------+-------+---------+
	Table 3		Table 3
	We can compare the median delay, and the delays observed by the		We can compare the median delay, and the delays observed by the
	fastest and slowest quartiles in the three years:		fastest and slowest quartiles in the three years:
	+======+=============+========+=============+		+======+=============+========+=============+
	| Year | Fastest 25% | Median | Slowest 25% |		| Year | Fastest 25% | Median | Slowest 25% |
	+======+=============+========+=============+		+======+=============+========+=============+
	| 2018 | 604 | 1179 | 1522 |		| 2018 | 715 | 1221 | 1537 |
	+------+-------------+--------+-------------+		+------+-------------+--------+-------------+
	| 2008 | 869 | 1081 | 1675 |		| 2008 | 869 | 1081 | 1675 |
	+------+-------------+--------+-------------+		+------+-------------+--------+-------------+
	| 1998 | 169 | 365 | 442 |		| 1998 | 169 | 365 | 442 |
	+------+-------------+--------+-------------+		+------+-------------+--------+-------------+
	Table 4		Table 4
	The IETF takes three to four times more to produce an RFC in 2018		The IETF takes three to four times more to produce an RFC in 2018
	than it did in 1998, but about the same time as it did in 2008. We		than it did in 1998, but about the same time as it did in 2008. We
	skipping to change at line 1254 ¶		skipping to change at line 1254 ¶
	| | | | | | | (total) |		| | | | | | | (total) |
	+=======+========+=======+==========+========+=========+=========+		+=======+========+=======+==========+========+=========+=========+
	| 8411 | Info | 5 | 53 | 88 | 20 | 161 |		| 8411 | Info | 5 | 53 | 88 | 20 | 161 |
	+-------+--------+-------+----------+--------+---------+---------+		+-------+--------+-------+----------+--------+---------+---------+
	| 8456 | Info | 64 | 98 | 46 | 14 | 158 |		| 8456 | Info | 64 | 98 | 46 | 14 | 158 |
	+-------+--------+-------+----------+--------+---------+---------+		+-------+--------+-------+----------+--------+---------+---------+
	| 8446 | PS | 160 | 85 | 57 | 0 | 142 |		| 8446 | PS | 160 | 85 | 57 | 0 | 142 |
	+-------+--------+-------+----------+--------+---------+---------+		+-------+--------+-------+----------+--------+---------+---------+
	| 8355 | Info | 13 | 83 | 15 | 1 | 99 |		| 8355 | Info | 13 | 83 | 15 | 1 | 99 |
	+-------+--------+-------+----------+--------+---------+---------+		+-------+--------+-------+----------+--------+---------+---------+
	| 8441 | PS | 8 | 67 | 33 | 6 | 106 |		| 8441 | PS | 8 | 56 | 33 | 3 | 92 |
	+-------+--------+-------+----------+--------+---------+---------+		+-------+--------+-------+----------+--------+---------+---------+
	| 8324 | Info | 29 | 42 | 28 | 1 | 71 |		| 8324 | Info | 29 | 42 | 28 | 1 | 71 |
	| | (ISE) | | | | | |		| | (ISE) | | | | | |
	+-------+--------+-------+----------+--------+---------+---------+		+-------+--------+-------+----------+--------+---------+---------+
	| 8377 | PS | 8 | 39 | 102 | 0 | 141 |		| 8377 | PS | 8 | 39 | 102 | 0 | 141 |
	+-------+--------+-------+----------+--------+---------+---------+		+-------+--------+-------+----------+--------+---------+---------+
	| 8498 | Info | 15 | 49 | 16 | 2 | 67 |		| 8498 | Info | 15 | 48 | 16 | 1 | 65 |
	+-------+--------+-------+----------+--------+---------+---------+		+-------+--------+-------+----------+--------+---------+---------+
	| 8479 | Info | 8 | 31 | 5 | 1 | 37 |		| 8479 | Info | 8 | 31 | 5 | 1 | 37 |
	| | (ISE) | | | | | |		| | (ISE) | | | | | |
	+-------+--------+-------+----------+--------+---------+---------+		+-------+--------+-------+----------+--------+---------+---------+
	| 8453 | Info | 42 | 73 | 7 | 0 | 80 |		| 8453 | Info | 42 | 73 | 7 | 3 | 83 |
	+-------+--------+-------+----------+--------+---------+---------+		+-------+--------+-------+----------+--------+---------+---------+
	| 8429 | BCP | 10 | 60 | 99 | 0 | 159 |		| 8429 | BCP | 10 | 60 | 99 | 0 | 159 |
	+-------+--------+-------+----------+--------+---------+---------+		+-------+--------+-------+----------+--------+---------+---------+
	| 8312 | Info | 18 | 96 | 30 | 0 | 126 |		| 8312 | Info | 18 | 55 | 28 | 2 | 85 |
	+-------+--------+-------+----------+--------+---------+---------+		+-------+--------+-------+----------+--------+---------+---------+
	| 8492 | Info | 40 | 355 | 123 | 2 | 480 |		| 8492 | Info | 40 | 355 | 123 | 2 | 480 |
	| | (ISE) | | | | | |		| | (ISE) | | | | | |
	+-------+--------+-------+----------+--------+---------+---------+		+-------+--------+-------+----------+--------+---------+---------+
	| 8378 | Exp | 21 | 42 | 9 | 0 | 51 |		| 8378 | Exp | 21 | 42 | 9 | 0 | 51 |
	+-------+--------+-------+----------+--------+---------+---------+		+-------+--------+-------+----------+--------+---------+---------+
	| 8361 | PS | 17 | 39 | 31 | 3 | 73 |		| 8361 | PS | 17 | 39 | 31 | 3 | 73 |
	+-------+--------+-------+----------+--------+---------+---------+		+-------+--------+-------+----------+--------+---------+---------+
	| 8472 | PS | 8 | 59 | 8 | 13 | 80 |		| 8472 | PS | 8 | 59 | 8 | 13 | 80 |
	+-------+--------+-------+----------+--------+---------+---------+		+-------+--------+-------+----------+--------+---------+---------+
	| 8471 | PS | 18 | 59 | 8 | 13 | 80 |		| 8471 | PS | 18 | 59 | 8 | 13 | 80 |
	+-------+--------+-------+----------+--------+---------+---------+		+-------+--------+-------+----------+--------+---------+---------+
	| 8466 | PS | 158 | 84 | 22 | 3 | 109 |		| 8466 | PS | 158 | 84 | 22 | 3 | 109 |
	+-------+--------+-------+----------+--------+---------+---------+		+-------+--------+-------+----------+--------+---------+---------+
	| 8362 | PS | 33 | 49 | 11 | 4 | 64 |		| 8362 | PS | 33 | 49 | 11 | 4 | 64 |
	+-------+--------+-------+----------+--------+---------+---------+		+-------+--------+-------+----------+--------+---------+---------+
	| 8468 | Info | 15 | 65 | 53 | 9 | 127 |		| 8468 | Info | 15 | 65 | 53 | 9 | 127 |
	+-------+--------+-------+----------+--------+---------+---------+		+-------+--------+-------+----------+--------+---------+---------+
	| Average | | 76 | 40 | 5 | 121 |		| Average | | 74 | 39 | 5 | 118 |
	+----------------+-------+----------+--------+---------+---------+		+----------------+-------+----------+--------+---------+---------+
	| Average | | 62 | 35 | 5 | 102 |		| Average | | 59 | 35 | 5 | 99 |
	| (without 8492) | | | | | |		| (without 8492) | | | | | |
	+----------------+-------+----------+--------+---------+---------+		+----------------+-------+----------+--------+---------+---------+
	Table 7		Table 7
	On average, the total delay appears to be about four months, but the		On average, the total delay appears to be about four months, but the
	average is skewed by the extreme values encountered for [RFC8492].		average is skewed by the extreme values encountered for [RFC8492].
	If we exclude that RFC from the computations, the average delay drops		If we exclude that RFC from the computations, the average delay drops
	to a just a bit more than 3 months: about 2 months for the		to a just a bit more than 3 months: about 2 months for the
	preparation, a bit more than one month for the AUTH48 phase, and 5		preparation, a bit more than one month for the AUTH48 phase, and 5
	skipping to change at line 1331 ¶		skipping to change at line 1331 ¶
	+===================+==========+========+=============+		+===================+==========+========+=============+
	| Correlation | RFC edit | AUTH48 | Edit(total) |		| Correlation | RFC edit | AUTH48 | Edit(total) |
	+===================+==========+========+=============+		+===================+==========+========+=============+
	| Number of pages | 0.50 | -0.04 | 0.21 |		| Number of pages | 0.50 | -0.04 | 0.21 |
	+-------------------+----------+--------+-------------+		+-------------------+----------+--------+-------------+
	| Copy-Edit | 0.42 | 0.24 | 0.45 |		| Copy-Edit | 0.42 | 0.24 | 0.45 |
	+-------------------+----------+--------+-------------+		+-------------------+----------+--------+-------------+
	| IANA | -0.14 | -0.21 | 0.12 |		| IANA | -0.14 | -0.21 | 0.12 |
	+-------------------+----------+--------+-------------+		+-------------------+----------+--------+-------------+
	| Number of Authors | 0.39 | -0.07 | 0.18 |		| Number of authors | 0.39 | -0.07 | 0.18 |
	+-------------------+----------+--------+-------------+		+-------------------+----------+--------+-------------+
	| Number of drafts | 0.18 | -0.33 | -0.19 |		| Number of drafts | 0.18 | -0.33 | -0.19 |
	+-------------------+----------+--------+-------------+		+-------------------+----------+--------+-------------+
	| WG delay | 0.03 | -0.16 | -0.15 |		| WG delay | 0.03 | -0.16 | -0.15 |
	+-------------------+----------+--------+-------------+		+-------------------+----------+--------+-------------+
	Table 8		Table 8
	We see some plausible explanations for the production delay. It will		We see some plausible explanations for the production delay. It will
	be somewhat longer for longer documents or for documents that require		be somewhat longer for longer documents or for documents that require
	skipping to change at line 1808 ¶		skipping to change at line 1808 ¶
	This protocol does not carry a significant proportion of Internet		This protocol does not carry a significant proportion of Internet
	traffic, but has been the object of a fair number of academic		traffic, but has been the object of a fair number of academic
	studies.		studies.
	The citation process tends to privilege the first expression of a		The citation process tends to privilege the first expression of a
	concept. We see that with the most cited RFC in the 1998 set is		concept. We see that with the most cited RFC in the 1998 set is
	[RFC2267], an informational RFC defining Network Ingress Filtering		[RFC2267], an informational RFC defining Network Ingress Filtering
	that was obsoleted in May 2000 by [RFC2827]. It is still cited		that was obsoleted in May 2000 by [RFC2827]. It is still cited
	frequently in 2018 and 2019, regardless of its formal status in the		frequently in 2018 and 2019, regardless of its formal status in the
	RFC Series. We see the same effect at work with [RFC8441], which		RFC Series. We see the same effect at work with [RFC8441], which
	garners very few citations although it obsoletes [RFC6455] that has a		garners very few citations although it updates [RFC6455] that has a
	large number of citations. The same goes for [RFC8468], which is		large number of citations. The same goes for [RFC8468], which is
	sparsely cited while the [RFC2330] is widely cited. Just counting		sparsely cited while the [RFC2330] is widely cited. Just counting
	citations will not indicate whether developers still use an old		citations will not indicate whether developers still use an old
	specification or have adopted the revised RFC.		specification or have adopted the revised RFC.
	5.4. Citations versus Web References		5.4. Citations versus Web References
	Web references might be another indicator of the popularity of an		Web references might be another indicator of the popularity of an
	RFC. In order to evaluate these references, we list here the number		RFC. In order to evaluate these references, we list here the number
	of results returned by searches on Google and Bing, looking for the		of results returned by searches on Google and Bing, looking for the
	skipping to change at line 2173 ¶		skipping to change at line 2173 ¶
	Many thanks to the authors of the selected RFCs who were willing to		Many thanks to the authors of the selected RFCs who were willing to
	provide feedback on the process: Michael Ackermann, Zafar Ali, Sarah		provide feedback on the process: Michael Ackermann, Zafar Ali, Sarah
	Banks, Bruno Decraene, Lars Eggert, Nalini Elkins, Joachim Fabini,		Banks, Bruno Decraene, Lars Eggert, Nalini Elkins, Joachim Fabini,
	Dino Farinacci, Clarence Filsfils, Sujay Gupta, Dan Harkins, Vinayak		Dino Farinacci, Clarence Filsfils, Sujay Gupta, Dan Harkins, Vinayak
	Hegde, Benjamin Kaduk, John Klensin, Acee Lindem, Nikos		Hegde, Benjamin Kaduk, John Klensin, Acee Lindem, Nikos
	Mavrogiannopoulos, Patrick McManus, Victor Moreno, Al Morton, Andrei		Mavrogiannopoulos, Patrick McManus, Victor Moreno, Al Morton, Andrei
	Popov, Eric Rescorla, Michiko Short, Bhuvaneswaran Vengainathan, Lao		Popov, Eric Rescorla, Michiko Short, Bhuvaneswaran Vengainathan, Lao
	Weiguo, and Li Yizhou. Many thanks to Adrian Farrel for his useful		Weiguo, and Li Yizhou. Many thanks to Adrian Farrel for his useful
	advice, to Stephen Farrell and Colin Perkins for their guidance on		advice, to Stephen Farrell and Colin Perkins for their guidance on
	the use of citations, and to Dave Crocker for a comprehensive review.		the use of citations, and to Dave Crocker for a comprehensive review.
			Thanks also to Alice Russo and the RFC Editor team for their work
			improving this document and checking the accuracy of the data.
	Author's Address		Author's Address
	Christian Huitema		Christian Huitema
	Private Octopus Inc.		Private Octopus Inc.
	427 Golfcourse Rd		427 Golfcourse Rd
	Friday Harbor, WA 98250		Friday Harbor, WA 98250
	United States of America		United States of America
	Email: huitema@huitema.net		Email: huitema@huitema.net
End of changes. 12 change blocks.
	11 lines changed or deleted		13 lines changed or added
This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/