Normal view

There are new articles available, click to refresh the page.
Before yesterdayThe Go

Go Turns 15

11 November 2024 at 07:00

The Go Blog

Go Turns 15

Austin Clements, for the Go team
11 November 2024


Thanks to Renee French for drawing and animating the gopher doing the “15 puzzle”.

Happy birthday, Go!

On Sunday, we celebrated the 15th anniversary of the Go open source release!

So much has changed since Go’s 10 year anniversary, both in Go and in the world. In other ways, so much has stayed the same: Go remains committed to stability, safety, and supporting software engineering and production at scale.

And Go is going strong! Go’s user base has more than tripled in the past five years, making it one of the fastest growing languages. From its beginnings just fifteen years ago, Go has become a top 10 language and the language of the modern cloud.

With the releases of Go 1.22 in February and Go 1.23 in August, it’s been the year of for loops. Go 1.22 made variables introduced by for loops scoped per iteration, rather than per loop, addressing a long-standing language “gotcha”. Over ten years ago, leading up to the release of Go 1, the Go team made decisions about several language details; among them whether for loops should create a new loop variable on each iteration. Amusingly, the discussion was quite brief and distinctly unopinionated. Rob Pike closed it out in true Rob Pike fashion with a single word: “stet” (leave it be). And so it was. While seemingly insignificant at the time, years of production experience highlighted the implications of this decision. But in that time, we also built robust tools for understanding the effects of changes to Go—notably, ecosystem-wide analysis and testing across the entire Google codebase—and established processes for working with the community and getting feedback. Following extensive testing, analysis, and community discussion, we rolled out the change, accompanied by a hash bisection tool to assist developers in pinpointing code affected by the change at scale.

The change to for loops was part of a five year trajectory of measured changes. It would not have been possible without forward language compatibility introduced in Go 1.21. This, in turn, built upon the foundation laid by Go modules, which were introduced in Go 1.14 four and a half years ago.

Go 1.23 further built on this change to introduce iterators and user-defined for-range loops. Combined with generics—introduced in Go 1.18, just two and a half years ago!—this creates a powerful and ergonomic foundation for custom collections and many other programming patterns.

These releases have also brought many improvements in production readiness, including much-anticipated enhancements to the standard library’s HTTP router, a total overhaul of execution traces, and stronger randomness for all Go applications. Additionally, the introduction of our first v2 standard library package establishes a template for future library evolution and modernization.

Over the past year we’ve also been cautiously rolling out opt-in telemetry for Go tools. This system will give Go’s developers data to make better decisions, while remaining completely open and anonymous. Go telemetry first appeared in gopls, the Go language server, where it has already led to a litany of improvements. This effort paves the way to make programming in Go an even better experience for everyone.

Looking forward, we’re evolving Go to better leverage the capabilities of current and future hardware. Hardware has changed a lot in the past 15 years. In order to ensure Go continues to support high-performance, large-scale production workloads for the next 15 years, we need to adapt to large multicores, advanced instruction sets, and the growing importance of locality in increasingly non-uniform memory hierarchies. Some of these improvements will be transparent. Go 1.24 will have a totally new map implementation under the hood that’s more efficient on modern CPUs. And we’re prototyping new garbage collection algorithms designed around the capabilities and constraints of modern hardware. Some improvements will be in the form of new APIs and tools so Go developers can better leverage modern hardware. We’re looking at how to support the latest vector and matrix hardware instructions, and multiple ways that applications can build in CPU and memory locality. A core principle guiding our efforts is composable optimization: the impact of an optimization on a codebase should be as localized as possible, ensuring that the ease of development across the rest of the codebase is not compromised.

We’re continuing to ensure Go’s standard library is safe by default and safe by design. This includes ongoing efforts to incorporate built-in, native support for FIPS-certified cryptography, so that FIPS crypto will be just a flag flip away for applications that need it. Furthermore, we’re evolving Go’s standard library packages where we can and, following the example of math/rand/v2, considering where new APIs can significantly enhance the ease of writing safe and secure Go code.

We’re working on making Go better for AI—and AI better for Go—by enhancing Go’s capabilities in AI infrastructure, applications, and developer assistance. Go is a great language for building production systems, and we want it to be a great language for building production AI systems, too. Go’s dependability as a language for Cloud infrastructure has made it a natural choice for LLM infrastructure as well. For AI applications, we will continue building out first-class support for Go in popular AI SDKs, including LangChainGo and Genkit. And from its very beginning, Go aimed to improve the end-to-end software engineering process, so naturally we’re looking at bringing the latest tools and techniques from AI to bear on reducing developer toil, leaving more time for the fun stuff—like actually programming!

Thank you

All of this is only possible because of Go’s incredible contributors and thriving community. Fifteen years ago we could only dream of the success that Go has become and the community that has developed around Go. Thank you to everyone who has played a part, large and small. We wish you all the best in the coming year.

What's in an (Alias) Name?

17 September 2024 at 07:00

The Go Blog

What's in an (Alias) Name?

Robert Griesemer
17 September 2024

This post is about generic alias types, what they are, and why we need them.

Background

Go was designed for programming at scale. Programming at scale means dealing with large amounts of data, but also large codebases, with many engineers working on those codebases over long periods of time.

Go’s organization of code into packages enables programming at scale by splitting up large codebases into smaller, more manageable pieces, often written by different people, and connected through public APIs. In Go, these APIs consist of the identifiers exported by a package: the exported constants, types, variables, and functions. This includes the exported fields of structs and methods of types.

As software projects evolve over time or requirements change, the original organization of the code into packages may turn out to be inadequate and require refactoring. Refactoring may involve moving exported identifiers and their respective declarations from an old package to a new package. This also requires that any references to the moved declarations must be updated so that they refer to the new location. In large codebases it may be unpractical or infeasible to make such a change atomically; or in other words, to do the move and update all clients in a single change. Instead, the change must happen incrementally: for instance, to “move” a function F, we add its declaration in a new package without deleting the original declaration in the old package. This way, clients can be updated incrementally, over time. Once all callers refer to F in the new package, the original declaration of F may be safely deleted (unless it must be retained indefinitely, for backward compatibility). Russ Cox describes refactoring in detail in his 2016 article on Codebase Refactoring (with help from Go).

Moving a function F from one package to another while also retaining it in the original package is easy: a wrapper function is all that’s needed. To move F from pkg1 to pkg2, pkg2 declares a new function F (the wrapper function) with the same signature as pkg1.F, and pkg2.F calls pkg1.F. New callers may call pkg2.F, old callers may call pkg1.F, yet in both cases the function eventually called is the same.

Moving constants is similarly straightforward. Variables take a bit more work: one may have to introduce a pointer to the original variable in the new package or perhaps use accessor functions. This is less ideal, but at least it is workable. The point here is that for constants, variables, and functions, existing language features exist that permit incremental refactoring as described above.

But what about moving a type?

In Go, the (qualified) identifier, or just name for short, determines the identity of types: a type T defined and exported by a package pkg1 is different from an otherwise identical type definition of a type T exported by a package pkg2. This property complicates a move of T from one package to another while retaining a copy of it in the original package. For instance, a value of type pkg2.T is not assignable to a variable of type pkg1.T because their type names and thus their type identities are different. During an incremental update phase, clients may have values and variables of both types, even though the programmer’s intent is for them to have the same type.

To solve this problem, Go 1.9 introduced the notion of a type alias. A type alias provides a new name for an existing type without introducing a new type that has a different identity.

In contrast to a regular type definition

type T T0

which declares a new type that is never identical to the type on the right-hand side of the declaration, an alias declaration

type A = T  // the "=" indicates an alias declaration

declares only a new name A for the type on the right-hand side: here, A and T denote the same and thus identical type T.

Alias declarations make it possible to provide a new name (in a new package!) for a given type while retaining type identity:

package pkg2

import "path/to/pkg1"

type T = pkg1.T

The type name has changed from pkg1.T to pkg2.T but values of type pkg2.T have the same type as variables of type pkg1.T.

Generic alias types

Go 1.18 introduced generics. Since that release, type definitions and function declarations can be customized through type parameters. For technical reasons, alias types didn’t gain the same ability at that time. Obviously, there were also no large codebases exporting generic types and requiring refactoring.

Today, generics have been around for a couple of years, and large codebases are making use of generic features. Eventually the need will arise to refactor these codebases, and with that the need to migrate generic types from one package to another.

To support incremental refactorings involving generic types, the future Go 1.24 release, planned for early February 2025, will fully support type parameters on alias types in accordance with proposal #46477. The new syntax follows the same pattern as it does for type definitions and function declarations, with an optional type parameter list following the identifier (the alias name) on the left-hand side. Before this change one could only write:

type Alias = someType

but now we can also declare type parameters with the alias declaration:

type Alias[P1 C1, P2 C2] = someType

Consider the previous example, now with generic types. The original package pkg1 declared and exported a generic type G with a type parameter P that is suitably constrained:

package pkg1

type Constraint      someConstraint
type G[P Constraint] someType

If the need arises to provide access to the same type G from a new package pkg2, a generic alias type is just the ticket (playground):

package pkg2

import "path/to/pkg1"

type Constraint      = pkg1.Constraint  // pkg1.Constraint could also be used directly in G
type G[P Constraint] = pkg1.G[P]

Note that one cannot simply write

type G = pkg1.G

for a couple of reasons:

  1. Per existing spec rules, generic types must be instantiated when they are used. The right-hand side of the alias declaration uses the type pkg1.G and therefore type arguments must be provided. Not doing so would require an exception for this case, making the spec more complicated. It is not obvious that the minor convenience is worth the complication.

  2. If the alias declaration doesn’t need to declare its own type parameters and instead simply “inherits” them from the aliased type pkg1.G, the declaration of G provides no indication that it is a generic type. Its type parameters and constraints would have to be retrieved from the declaration of pkg1.G (which itself might be an alias). Readability will suffer, yet readable code is one of the primary aims of the Go project.

Writing down an explicit type parameter list may seem like an unnecessary burden at first, but it also provides additional flexibility. For one, the number of type parameters declared by the alias type doesn’t have to match the number of type parameters of the aliased type. Consider a generic map type:

type Map[K comparable, V any] mapImplementation

If uses of Map as sets are common, the alias

type Set[K comparable] = Map[K, bool]

might be useful (playground). Because it is an alias, types such as Set[int] and Map[int, bool] are identical. This would not be the case if Set were a defined (non-alias) type.

Furthermore, the type constraints of a generic alias type don’t have to match the constraints of the aliased type, they only have to satisfy them. For instance, reusing the set example above, one could define an IntSet as follows:

type integers interface{ ~int | ~int8 | ~int16 | ~int32 | ~int64 }
type IntSet[K integers] = Set[K]

This map can be instantiated with any key type that satisfies the integers constraint (playground). Because integers satisfies comparable, the type parameter K may be used as type argument for the K parameter of Set, following the usual instantiation rules.

Finally, because an alias may also denote a type literal, parameterized aliases make it possible to create generic type literals (playground):

type Point3D[E any] = struct{ x, y, z E }

To be clear, none of these examples are “special cases” or somehow require additional rules in the spec. They follow directly from the application of the existing rules put in place for generics. The only thing that changed in the spec is the ability to declare type parameters in an alias declaration.

An interlude about type names

Before the introduction of alias types, Go had only one form of type declarations:

type TypeName existingType

This declaration creates a new and different type from an existing type and gives that new type a name. It was natural to call such types named types as they have a type name in contrast to unnamed type literals such as struct{ x, y int }.

With the introduction of alias types in Go 1.9 it became possible to give a name (an alias) to type literals, too. For instance, consider:

type Point2D = struct{ x, y int }

Suddenly, the notion of a named type describing something that is different from a type literal didn’t make that much sense anymore, since an alias name clearly is a name for a type, and thus the denoted type (which might be a type literal, not a type name!) arguably could be called a “named type”.

Because (proper) named types have special properties (one can bind methods to them, they follow different assignment rules, etc.), it seemed prudent to use a new term in order to avoid confusions. Thus, since Go 1.9, the spec calls the types formerly called named types defined types: only defined types have properties (methods, assignability restrictions, etc) that are tied to their names. Defined types are introduced through type definitions, and alias types are introduced through alias declarations. In both cases, names are given to types.

The introduction of generics in Go 1.18 made things more complicated. Type parameters are types, too, they have a name, and they share rules with defined types. For instance, like defined types, two differently named type parameters denote different types. In other words, type parameters are named types, and furthermore, they behave similarly to Go’s original named types in some ways.

To top things off, Go’s predeclared types (int, string and so on) can only be accessed through their names, and like defined types and type parameters, are different if their names are different (ignoring for a moment the byte and rune alias types). The predeclared types truly are named types.

Therefore, with Go 1.18, the spec came full circle and formally re-introduced the notion of a named type which now comprises “predeclared types, defined types, and type parameters”. To correct for alias types denoting type literals the spec says: “An alias denotes a named type if the type given in the alias declaration is a named type.”

Stepping back and outside the box of Go nomenclature for a moment, the correct technical term for a named type in Go is probably nominal type. A nominal type’s identity is explicitly tied to its name which is exactly what Go’s named types (now using the 1.18 terminology) are all about. A nominal type’s behavior is in contrast to a structural type which has behavior that only depends on its structure and not its name (if it has one in the first place). Putting it all together, Go’s predeclared, defined, and type parameter types are all nominal types, while Go’s type literals and aliases denoting type literals are structural types. Both nominal and structural types can have names, but having a name doesn’t mean the type is nominal, it just means it is named.

None of this matters for day-to-day use of Go and in practice the details can safely be ignored. But precise terminology matters in the spec because it makes it easier to describe the rules governing the language. So should the spec change its terminology one more time? It is probably not worth the churn: it is not just the spec that would need to be updated, but also a lot of supporting documentation. A fair number of books written on Go might become inaccurate. Furthermore, “named”, while less precise, is probably intuitively clearer than “nominal” for most people. It also matches the original terminology used in the spec, even if it now requires an exception for alias types denoting type literals.

Availability

Implementing generic type aliases has taken longer than expected: the necessary changes required adding a new exported Alias type to go/types and then adding the ability to record type parameters with that type. On the compiler side, the analogous changes also required modifications to the export data format, the file format that describes a package’s exports, which now needs to be able to describe type parameters for aliases. The impact of these changes is not confined to the compiler, but affects clients of go/types and thus many third-party packages. This was very much a change affecting a large code base; to avoid breaking things, an incremental roll-out over several releases was necessary.

After all this work, generic alias types will finally be available by default in Go 1.24.

To allow third-party clients to get their code ready, starting with Go 1.23, support for generic type aliases can be enabled by setting GOEXPERIMENT=aliastypeparams when invoking the go tool. However, be aware that support for exported generic aliases is still missing for that version.

Full support (including export) is implemented at tip, and the default setting for GOEXPERIMENT will soon be switched so that generic type aliases are enabled by default. Thus, another option is to experiement with the latest version of Go at tip.

As always, please let us know if you encounter any problems by filing an issue; the better we test a new feature, the smoother the general roll-out.

Thanks and happy refactoring!

❌
❌