Home Backend Development Golang Go-DOM - A headless browser written in Go.

Go-DOM - A headless browser written in Go.

Nov 07, 2024 pm 06:28 PM

Go-DOM - A headless browser written in Go.

Having too little to do sometimes results in a crazy idea, and this time; it was to write a headless browser in Go, with full DOM implementation and JavaScript support, by embedding the v8 engine.

It all started working with writing an HTMX application, and the need to test it that made me curious if there was a pure Go implementation of a headless browsser.

Searching for "go headless browser" only resulted in search results talking about automating a headless browser, i.e. using a real browser like Chrome of Firefox in headless mode.

But nothing in pure Go.

So I started building one.

Why A Headless Browser in Go?

It may seem silly because writing a headless browser will never work like a real browser; and as such wouldn't really verify that your application works correctly in all the browsers you have decided to support. Neither does this allow you to get nice features such as screenshots of the application when things stop working.

So why then?

The Need for Speed!

To work in an effective TDD loop, tests must be fast. Slow test execution discourages TDD, and you loose the efficiency benefits a fast feedback loop provides.

Using browser automation for this type of verification has severe overheads, and such tests are typically written after the code was written; and as such, they no longer serve as a help writing the correct implementation; but are reduced to a maintenance burden after the fact; that only occasionally detect a bug before your paying customers do.

The goal is to create a tool that supports a TDD process. To be usable, it needs to run in-process.

It needs to be written in Go.

Less Flaky Tests

Having the DOM in-process enables writing better wrappers on top of the DOM; which can help providing a less erratic interface for your tests, like testing-library does for JavaScript.

Rather than depending on CSS classnames, element IDs, or DOM structure, you write your tests in a user-centric language, like this.

Type "me@example.com" in the textbox that has the label, "Email"

Or in hypothetical code.

testing.GetElement(Query{
  role: "textbox",
  // The accessibility "name" of a textbox _is_ the label
  name: "Email",
}).type("me@example.com")
Copy after login
Copy after login

This test doesn't care if the label is implemented as

This decouples verification of behaviour from UI changes; but it does enforce that the text, "Email" is associated with the input field in accessible way. This couples the test to how the user interacts with the page; including those relying on screen readers for using your page.

This achieves the most important aspect of TDD; to write tests coupled to concrete behaviour.1

Although it's probably technically possible to write the same tests for an out-of-process browser; the benefit of native code is essential for the type of random access of the DOM you most likely need for these types of helpers.

An example: JavaScript

To exemplify the type of test, I will use a similar example from JavaScript; also an application using HTMX. The test verifies a general login flow from requesting a page requiring authentication.

It's a bit long, as I've combined all setup and helper code in one test function here.

testing.GetElement(Query{
  role: "textbox",
  // The accessibility "name" of a textbox _is_ the label
  name: "Email",
}).type("me@example.com")
Copy after login
Copy after login

In simple terms the test does the following:

  1. Stub out the authentication function, simulating a successful response.
  2. Request a page that requires authentication
  3. Verify that the browser redirects to the login page, and the browser URL is updated. 2
  4. Fill out the form with the expected values, and submit.
  5. Verify that the browser redirects to the originally requested page, and it shows information for the stubbed user.

Internally the test starts an HTTP server. Because the this runs in the test process, mocking and stubbing of business logic is possible. The test use jsdom to communicate with the HTTP server; which both parse the HTML response into a DOM, but also executes client-side script in a sandbox which has been initialised, e.g. with window as the global scope.3

This enables writing tests of the HTTP layer, where validating the contents of the response is not enough. In this case; that the response is processed by HTMX as intended.

But apart from waiting for some HTMX events, so as to not proceed to early (or too late) the test doesn't actually care about HTMX. In fact, if I remove HTMX from the form, resorting to classical redirects, the test still pass.

(If I remove the HTMX

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Hot Topics

Java Tutorial
1664
14
PHP Tutorial
1268
29
C# Tutorial
1240
24
Golang's Purpose: Building Efficient and Scalable Systems Golang's Purpose: Building Efficient and Scalable Systems Apr 09, 2025 pm 05:17 PM

Go language performs well in building efficient and scalable systems. Its advantages include: 1. High performance: compiled into machine code, fast running speed; 2. Concurrent programming: simplify multitasking through goroutines and channels; 3. Simplicity: concise syntax, reducing learning and maintenance costs; 4. Cross-platform: supports cross-platform compilation, easy deployment.

Golang vs. Python: Performance and Scalability Golang vs. Python: Performance and Scalability Apr 19, 2025 am 12:18 AM

Golang is better than Python in terms of performance and scalability. 1) Golang's compilation-type characteristics and efficient concurrency model make it perform well in high concurrency scenarios. 2) Python, as an interpreted language, executes slowly, but can optimize performance through tools such as Cython.

Golang and C  : Concurrency vs. Raw Speed Golang and C : Concurrency vs. Raw Speed Apr 21, 2025 am 12:16 AM

Golang is better than C in concurrency, while C is better than Golang in raw speed. 1) Golang achieves efficient concurrency through goroutine and channel, which is suitable for handling a large number of concurrent tasks. 2)C Through compiler optimization and standard library, it provides high performance close to hardware, suitable for applications that require extreme optimization.

Golang's Impact: Speed, Efficiency, and Simplicity Golang's Impact: Speed, Efficiency, and Simplicity Apr 14, 2025 am 12:11 AM

Goimpactsdevelopmentpositivelythroughspeed,efficiency,andsimplicity.1)Speed:Gocompilesquicklyandrunsefficiently,idealforlargeprojects.2)Efficiency:Itscomprehensivestandardlibraryreducesexternaldependencies,enhancingdevelopmentefficiency.3)Simplicity:

Golang vs. Python: Key Differences and Similarities Golang vs. Python: Key Differences and Similarities Apr 17, 2025 am 12:15 AM

Golang and Python each have their own advantages: Golang is suitable for high performance and concurrent programming, while Python is suitable for data science and web development. Golang is known for its concurrency model and efficient performance, while Python is known for its concise syntax and rich library ecosystem.

Golang and C  : The Trade-offs in Performance Golang and C : The Trade-offs in Performance Apr 17, 2025 am 12:18 AM

The performance differences between Golang and C are mainly reflected in memory management, compilation optimization and runtime efficiency. 1) Golang's garbage collection mechanism is convenient but may affect performance, 2) C's manual memory management and compiler optimization are more efficient in recursive computing.

Golang vs. C  : Performance and Speed Comparison Golang vs. C : Performance and Speed Comparison Apr 21, 2025 am 12:13 AM

Golang is suitable for rapid development and concurrent scenarios, and C is suitable for scenarios where extreme performance and low-level control are required. 1) Golang improves performance through garbage collection and concurrency mechanisms, and is suitable for high-concurrency Web service development. 2) C achieves the ultimate performance through manual memory management and compiler optimization, and is suitable for embedded system development.

The Performance Race: Golang vs. C The Performance Race: Golang vs. C Apr 16, 2025 am 12:07 AM

Golang and C each have their own advantages in performance competitions: 1) Golang is suitable for high concurrency and rapid development, and 2) C provides higher performance and fine-grained control. The selection should be based on project requirements and team technology stack.

See all articles