Advanced neural networks in Keras: The Functional API

The Sequential model gets you a long way, but it has a hard ceiling. It can only describe a straight line of layers, input flowing through one transformation after another to a single output. The moment you need two input streams, or two predictions, or a single layer reused in different parts of the network, that linear stack cannot express what you want. The Functional API removes the ceiling by treating layers as functions you call on tensors, so the computation graph can branch, merge, and reuse layers freely. This article works through the patterns that unlock: multiple inputs, shared layers, model stacking, and a single network that predicts both a number and a category at once. The running example is a sports rating model, which turns out to be a beautiful way to see why these patterns matter.

How the Functional API thinks

The Functional API has four moves. You define a symbolic input, you call layers on tensors as if they were functions, you wrap the whole graph into a Model by naming its inputs and outputs, and then you compile and fit exactly as you always have.

from keras.layers import Input, Dense
from keras.models import Model
input_tensor = Input(shape=(1,))
output_tensor = Dense(1)(input_tensor)
model = Model(input_tensor, output_tensor)
model.compile(optimizer='adam', loss='mean_absolute_error')

A couple of ideas make this click. Input does not hold any data. It is the “enter here” arrow on a flowchart, a placeholder that tells Keras what shape to expect for a single sample, with the batch dimension handled automatically. And the curious double-parenthesis line, Dense(1)(input_tensor), is really two steps fused together. The first parentheses build the layer, like ordering a machine whose weights do not exist yet, and the second call it on a tensor, which is the moment Keras inspects the incoming shape, builds the weights, and wires the two nodes into the graph. Wrapping it in Model then draws a box around everything between the named input and output, tracing backward from the output to discover every layer in between and packaging them into one trainable object.

One small modelling choice worth noticing is the loss. Mean squared error optimises toward the mean and punishes large errors harshly, while mean absolute error optimises toward the median and shrugs off outliers. For predicting a score margin where the occasional blowout happens, mean absolute error is the more robust default.

Embeddings as learnable lookups

Now the interesting part. Suppose you want to predict the margin of a match from the two teams involved, and there are thousands of distinct teams. One-hot encoding each team would give every match two enormous sparse vectors, which is wasteful. An embedding layer is the elegant alternative: a trainable table with one row per team, where you hand it a team ID and it returns that team’s row.

from keras.layers import Embedding
team_lookup = Embedding(
    input_dim=n_teams,
    output_dim=1,
    input_length=1,
    name='Team-Strength',
)

The embedding is just a matrix of shape teams by dimensions, and its rows update during training like any other weights. If a team keeps winning, its row drifts upward. The choice of a single output dimension is deliberate here, because we are modelling one number per team, a learned strength rating, derived entirely from match outcomes. Real word embeddings use fifty to three hundred dimensions, but one is exactly right when the next step is to subtract two ratings.

The first trick of the chapter is to package that lookup into its own reusable mini-model.

from keras.layers import Flatten
teamid_in = Input(shape=(1,))
strength = team_lookup(teamid_in)
strength_flat = Flatten()(strength)
team_strength_model = Model(teamid_in, strength_flat, name='Team-Strength-Model')

The Flatten is there because an embedding always adds a sequence dimension, and even when its length is one, that extra axis is still present, so flattening drops it to leave a clean shape. The point of wrapping this as a Model is that you can now call team_strength_model like a layer anywhere in a larger network, and every call uses the same internal weights.

Sharing weights across symmetric inputs

This is the key insight of the whole approach. A match has two teams, and you want a single rating per team used consistently no matter which side of the fixture it appears on. You get that by calling the same strength model on both team inputs.

from keras.layers import Subtract
team_in_1 = Input((1,), name='Team-1-In')
team_in_2 = Input((1,), name='Team-2-In')
team_1_strength = team_strength_model(team_in_1)
team_2_strength = team_strength_model(team_in_2)
score_diff = Subtract()([team_1_strength, team_2_strength])
model = Model([team_in_1, team_in_2], score_diff)

Because team_strength_model is one object, both calls share literally the same weight matrix. When backpropagation raises a team’s rating after a win, that update applies whether the team appeared as the first or second input, which is exactly what a fair rating system demands. The Subtract layer then encodes a clean assumption: the predicted score difference is simply the first team’s strength minus the second’s, with no hidden layers and no non-linearity, the same algebra that underlies Elo ratings. A model with two inputs is declared by passing a list to Model, and you mirror that list at fit time, passing the two team-ID columns as a list so Keras routes each to its matching input by position.

The reason this design is worth the effort shows up when you evaluate. Train it on league matches, then test it on cup matches, which pit teams that often never met during the league season. A model that merely memorised specific fixtures would fall apart on those novel pairings. One that genuinely learned per-team strengths carries over, and that gap is the honest test of whether the network learned something real.

Concatenating when inputs play different roles

Subtraction works because two team strengths are symmetric, two measurements of the same kind of thing. But suppose you add a home-advantage flag. That input is not the same kind of quantity as a strength rating, so subtracting it would be meaningless. When inputs play different roles you concatenate instead, stacking them side by side into one wider vector, and let a dense layer learn how to weigh each.

from keras.layers import Concatenate
home_in = Input(shape=(1,), name='Home-In')
out = Concatenate()([team_1_strength, team_2_strength, home_in])
out = Dense(1)(out)
model = Model([team_in_1, team_in_2, home_in], out)

This is the general rule for combining tensors. Reach for Subtract, Add, or Average when the inputs are symmetric and you want their difference, sum, or mean. Reach for Concatenate when they are asymmetric and you want the model to discover the right combination, here effectively learning how many points home advantage is worth relative to raw strength. The plumbing is unchanged: add a third array to every list and Keras routes it to the third input by position.

Stacking models

Once you have a trained model, its predictions are themselves a useful feature. This is model stacking. Run the league-trained model over the cup data, store its output as a new column, and feed that column into a second, simpler model alongside other features.

cup_matches['pred'] = model.predict(
    [cup_matches['team_1'], cup_matches['team_2'], cup_matches['home']]
)
input_tensor = Input((3,))
output_tensor = Dense(1)(input_tensor)
stacked = Model(input_tensor, output_tensor)
stacked.compile(optimizer='adam', loss='mean_absolute_error')

That prediction column is a distilled summary of everything the first model learned about team strength and home advantage, compressed into one number per match. The second model starts from that summary rather than raw data, so it can be tiny, just a single linear layer combining the prediction with the ranking difference and the home flag. Notice also that when all your features are already numeric and on comparable scales, you do not need separate input streams at all. You stack them into one Input(shape=(3,)) and pass a three-column DataFrame slice directly, which Keras reads as a matrix. The one thing to watch with stacking is leakage: train the first model on different data than the second model’s training set, or the inherited predictions will make training error look misleadingly good.

More than one output

The Functional API makes multiple outputs just as natural as multiple inputs. To predict both teams’ scores at once rather than only their difference, you use two output neurons.

input_tensor = Input(shape=(2,))
output_tensor = Dense(2)(input_tensor)
model = Model(input_tensor, output_tensor)
model.compile(optimizer='adam', loss='mean_absolute_error')

The loss is applied to each output independently and then summed, so the network must find weights that satisfy both at once, which forces a consistent internal story about what drives scoring on both sides. Inspecting the learned weights afterward is genuinely illuminating. The biases tend to converge toward the average score of each output, and the ranking-difference weights come out opposite in sign for the two outputs, because a larger ranking gap should push the stronger team’s score up and the weaker team’s score down at the same time.

One network, two kinds of prediction

The most interesting pattern combines regression and classification in a single model with two heads, and chains them so the classification depends on the regression.

input_tensor = Input(shape=(2,))
output_tensor_1 = Dense(1, activation='linear', use_bias=False)(input_tensor)
output_tensor_2 = Dense(1, activation='sigmoid', use_bias=False)(output_tensor_1)
model = Model(input_tensor, [output_tensor_1, output_tensor_2])
model.compile(
    loss=['mean_absolute_error', 'binary_crossentropy'],
    optimizer=Adam(learning_rate=0.01),
)

The clever part is that the second output takes the first as its input, not the original features. The classification head never sees the raw data at all, only the predicted score difference, which encodes a belief that win probability is purely a function of expected margin. The sigmoid converts that continuous margin into a probability, and a single learned weight controls how steeply margins translate into win certainty. During training Keras computes both losses, the mean absolute error on the margin and the binary cross-entropy on the win, sums them, and backpropagates the total through both heads in one step. This division of labour is efficient because the regression signal is dense, giving a continuous error on every match, while the win signal is just a zero or one. By linking the heads, the regression task does the heavy lifting and the classification head only has to learn one number, the slope of the sigmoid.

That single weight is fully interpretable, which is a rare treat in deep learning. Plug it into the sigmoid and you can read off real probabilities: a one-point favourite might come out around fifty-three percent, while a ten-point favourite sits near eighty. When you evaluate a multi-output model, Keras returns a list that starts with the total loss followed by each output’s loss separately, which lets you diagnose the heads independently and see, for instance, whether the regression is converging cleanly while the classifier is still noisy.

The patterns worth remembering

Strip away the sports example and four reusable patterns remain. When two inputs are symmetric, the same kind of thing on each side, run them through one shared layer or sub-model and combine them with subtraction or addition, which is exactly the structure behind Siamese networks for comparing image pairs or matching questions to answers. When inputs are asymmetric, with different roles, types, or scales, concatenate them and let a dense layer learn how to weigh each. When one model has already compressed hard-won patterns into useful predictions, stack it by feeding those predictions as features into a simpler second model. And when you have several related targets, share a backbone that learns one common representation and attach lightweight heads that each interpret it differently, sometimes finding that a task hard to learn alone becomes easier when a related task pulls the shared representation in a helpful direction.

The decision is usually mechanical once you ask two questions. How many inputs, and are they symmetric or not, which chooses between a shared-and-subtract design and a concatenate-and-dense one. And how many outputs, with the activation on each head chosen by its task, linear for regression, softmax for mutually exclusive classes, sigmoid for independent labels, and a chain between heads when one prediction genuinely depends on another. Answer those and the Functional API gives you the freedom to build the model your problem actually needs rather than the one a linear stack allows.

See you soon.