Logits#
This tutorial shows you how to use logits. Logits are un-normalized log-probabilities of amino acids at each position according to the masked-language model.
What you need before getting started#
You need a protein sequence of interest.
Requesting logits#
Select your model:
[ ]:
model = session.embedding.get_model("prot-seq")
Defne your sequence and request logits, returning the final output layer of the language model. This example uses semaglutide:
[ ]:
sequence = "HXEGTFTSDVSSYLEGQAAKEFIAWLVRGRG"
logits_future = model.logits([sequence])
print(logits_future.job)
status=<JobStatus.PENDING: 'PENDING'> job_id='b9098e33-536c-4bcb-8159-4e63a5cf976d' job_type='/embeddings/logits' created_date=datetime.datetime(2024, 6, 13, 3, 16, 6, 215705, tzinfo=datetime.timezone.utc) start_date=None end_date=None prerequisite_job_id=None progress_message=None progress_counter=0 num_records=1 sequence_length=None
Wait until the results are ready, polling periodically and reporting job progress (if multiple sequences are sent in the inference request):
[ ]:
logits_future.wait_until_done(verbose=True)
Waiting: 100%|██████████| 100/100 [02:46<00:00, 1.67s/it, status=SUCCESS]
True
The results are displayed as a list of (sequence, np.ndarray) tuples. The list is not guaranteed to be in the same order as the input. We recommend checking the sequence in the returned results.
Get the results:
[ ]:
logits_results = logits_future.get() # or logits_future.wait()
returned_sequence, logits = logits_results[0]
print(returned_sequence)
HXEGTFTSDVSSYLEGQAAKEFIAWLVRGRG
Logits are of shape LxV, where V is the vocabulary size for the model. Logits are not softmaxed. If you require normalized probabilities, pass the logits through a softmax or log_softmax function.
[ ]:
print(logits.shape)
(31, 20)
[ ]:
logits
array([[ 2.9614e-01, 3.3838e-01, -7.8125e-01, -1.9043e-01, -1.3350e+00,
-5.0146e-01, -3.1586e-02, 3.4961e-01, -8.7695e-01, -4.4525e-02,
6.7139e-01, -2.2937e-01, 2.5391e+00, -1.9055e-01, -8.6792e-02,
-6.0394e-02, -6.9885e-02, -8.2275e-01, -5.4150e-01, 4.6606e-01],
[ 4.7363e-01, 3.4473e-01, -5.8057e-01, -2.8305e-02, -1.2930e+00,
-2.8247e-01, 2.3572e-01, 6.8848e-01, -4.5166e-01, 2.2571e-01,
7.9297e-01, -8.1909e-02, -9.6240e-01, 9.7107e-02, 6.4160e-01,
2.9150e-01, 9.7427e-03, -1.0557e+00, -3.7207e-01, 4.1333e-01],
[ 1.0547e+00, 9.2188e-01, -1.1168e-03, 6.3330e-01, -6.5869e-01,
8.5938e-02, 2.1741e-01, 1.0322e+00, 1.2909e-02, 4.0479e-01,
1.0459e+00, 2.4768e-01, -5.3711e-01, 3.0029e-01, 7.3535e-01,
8.5693e-01, 4.8120e-01, -4.3701e-01, -2.0584e-02, 6.3623e-01],
[ 2.7173e-01, 2.6489e-01, -5.3271e-01, -4.6509e-02, -1.2549e+00,
-3.2251e-01, 2.2693e-01, 4.6216e-01, -8.1592e-01, 5.2307e-02,
5.4004e-01, -4.9347e-02, -8.4521e-01, -2.8564e-01, -1.7685e-02,
-1.1459e-02, -2.0227e-01, -1.1328e+00, -5.6885e-01, 2.4194e-01],
[ 4.9097e-01, 7.0557e-01, -6.1084e-01, 2.5098e-01, -5.0439e-01,
-1.3843e-01, 4.1968e-01, 6.4697e-01, -2.9858e-01, 3.0615e-01,
8.1299e-01, -6.7017e-02, -4.4995e-01, 3.3545e-01, -1.6785e-01,
2.7271e-01, -1.0919e-01, -2.9956e-01, 5.4138e-02, 7.0215e-01],
[ 2.2620e-01, 2.5732e-01, -6.5869e-01, -1.7944e-01, -1.2383e+00,
-4.8779e-01, 7.7393e-02, 3.2080e-01, -8.1006e-01, 2.4854e-01,
5.7861e-01, 4.5121e-05, -9.1504e-01, -1.4978e-01, 2.9858e-01,
3.6816e-01, 1.4587e-01, -6.7480e-01, -1.5442e-01, 3.6523e-01],
[ 3.4595e-01, 4.5581e-01, -5.7959e-01, 2.1729e-01, -8.3447e-01,
-3.5840e-01, 2.3254e-01, 5.2588e-01, -4.2480e-01, 7.8369e-02,
5.8740e-01, -1.9873e-01, -8.2324e-01, 1.0608e-01, 3.9087e-01,
4.2261e-01, -1.1835e-01, -4.2139e-01, -1.9897e-01, 5.0244e-01],
[ 6.9775e-01, 4.4189e-01, -5.0098e-01, 4.9219e-01, -1.1299e+00,
-2.0920e-02, 7.7734e-01, 6.1084e-01, -4.8486e-01, 4.0601e-01,
7.6514e-01, 2.0020e-01, -5.2930e-01, 1.6309e-01, 2.5732e-01,
5.9662e-02, -1.6525e-02, -8.7158e-01, -1.7908e-01, 7.4365e-01],
[ 7.1436e-01, 4.6289e-01, -1.7175e-01, 3.2007e-01, -6.5430e-01,
8.8745e-02, 5.4590e-01, 3.9111e-01, -4.3628e-01, 3.5596e-01,
9.4580e-01, 1.9812e-01, -2.8711e-01, 2.6245e-01, 2.6147e-01,
5.3662e-01, 2.8027e-01, -3.0176e-01, -1.8726e-01, 5.6055e-01],
[ 5.1465e-01, 4.6899e-01, -3.8623e-01, 2.6172e-01, -8.7549e-01,
-1.1066e-01, 3.4399e-01, 4.8901e-01, -2.1619e-01, 3.2764e-01,
9.5264e-01, -4.4287e-01, -6.8018e-01, 2.3059e-01, 3.7329e-01,
2.9761e-01, -3.2623e-02, -1.7004e-01, -1.0785e-01, 3.5864e-01],
[ 6.4209e-01, 5.4590e-01, -6.5186e-01, 1.1816e-01, -1.0088e+00,
-2.3560e-01, 4.9536e-01, 1.2317e-01, -5.7178e-01, 1.8701e-01,
8.8232e-01, -2.2668e-01, -3.6182e-01, -1.0663e-01, 3.7079e-02,
1.1566e-01, -9.4971e-02, -6.2354e-01, -4.5898e-01, 5.8008e-01],
[ 8.5352e-01, 6.5186e-01, -6.5527e-01, 1.9702e-01, -1.2285e+00,
-1.9983e-01, 3.6450e-01, 3.4302e-01, -4.7778e-01, -1.5967e-01,
3.8159e-01, 9.1919e-02, -8.7207e-01, -3.6890e-01, 2.7222e-01,
2.0767e-02, -9.9731e-02, -1.0674e+00, -6.9629e-01, 2.7905e-01],
[ 5.7471e-01, 1.1957e-01, -7.9199e-01, -4.4556e-01, -1.0977e+00,
-4.1919e-01, 4.2969e-02, 9.6741e-03, -4.6777e-01, 2.5610e-01,
8.5498e-01, -4.2432e-01, -5.2979e-01, 3.5010e-01, -8.1970e-02,
-1.2802e-02, -2.4133e-01, -9.0759e-02, 2.4170e-01, 4.2896e-01],
[ 4.9536e-01, 3.4253e-01, -4.6826e-01, 1.9788e-01, -9.6875e-01,
-2.8223e-01, 4.7144e-01, 3.0664e-01, -7.8955e-01, 2.8003e-01,
8.3203e-01, -1.0175e-01, -3.3057e-01, -1.1737e-01, 3.5352e-01,
2.5977e-01, -6.6101e-02, -9.2432e-01, -6.0449e-01, 3.4717e-01],
[ 1.2178e+00, 7.4707e-01, 9.0637e-02, 5.7471e-01, -9.3213e-01,
1.8567e-01, 2.1997e-01, 4.9902e-01, -1.6711e-01, 2.1362e-01,
7.2461e-01, 2.6929e-01, -7.5586e-01, -1.8738e-01, 6.0254e-01,
7.9346e-01, 4.6899e-01, -7.4219e-01, -2.0471e-01, 5.2246e-01],
[ 3.7695e-01, 4.7241e-01, 2.2058e-01, 5.2441e-01, -1.2471e+00,
1.9751e-01, 6.7676e-01, 8.1299e-01, -5.4150e-01, -2.1362e-01,
2.4329e-01, 2.4194e-01, -9.4531e-01, -5.2295e-01, 3.8623e-01,
4.0063e-01, 2.4146e-01, -1.2656e+00, -5.8838e-01, 1.3562e-01],
[ 3.5474e-01, 5.4688e-01, -8.1201e-01, 2.0752e-01, -1.5439e+00,
-2.0126e-02, 7.0410e-01, -1.4600e-01, -9.5312e-01, -4.4922e-02,
5.2490e-01, 4.3066e-01, -5.3760e-01, -6.5869e-01, 2.5146e-01,
-2.7124e-01, -3.2837e-01, -1.2168e+00, -8.4180e-01, 2.3596e-01],
[ 8.0078e-01, 6.6504e-01, -2.8052e-01, 7.5391e-01, -8.6963e-01,
1.5869e-01, 9.2480e-01, 5.9082e-01, -3.8794e-01, 1.8042e-01,
6.3672e-01, 3.9697e-01, -5.3271e-01, -1.3354e-01, 1.2579e-03,
3.1689e-01, 1.1823e-01, -4.6802e-01, -2.7466e-01, 5.5176e-01],
[ 8.2764e-01, 6.7334e-01, -3.9209e-01, 8.1116e-02, -8.9160e-01,
2.9160e-02, 4.5483e-01, 7.9443e-01, -7.4902e-01, 2.8760e-01,
8.5059e-01, 1.6541e-01, -8.3496e-02, 5.6427e-02, 3.2422e-01,
2.1069e-01, 2.3911e-02, -3.2788e-01, -2.1851e-01, 7.9150e-01],
[ 1.1514e+00, 1.3477e+00, -3.5498e-01, 3.4155e-01, -9.3115e-01,
3.7549e-01, 4.4922e-01, 6.1279e-01, -9.0485e-03, -6.9946e-02,
6.3232e-01, 8.2397e-02, -7.0996e-01, -6.3281e-01, 7.0947e-01,
1.9580e-01, -1.4844e-01, -8.3252e-01, -8.0420e-01, 3.8232e-01],
[ 1.5967e+00, 1.3418e+00, -1.0419e-04, 1.3867e+00, -6.0791e-01,
4.1187e-01, 1.1494e+00, 6.7822e-01, -2.8979e-01, -1.9446e-01,
7.0947e-01, 5.7031e-01, -8.0713e-01, -6.2158e-01, 5.5713e-01,
5.3613e-01, 3.4106e-01, -6.1523e-01, -6.9336e-01, 3.1787e-01],
[ 6.8311e-01, 6.9238e-01, -7.3096e-01, -2.4902e-01, -1.1289e+00,
-2.2900e-01, 2.1997e-01, 1.3171e-01, -7.7686e-01, 3.0981e-01,
6.0645e-01, 7.3792e-02, -2.6904e-01, -1.6724e-01, -6.0205e-01,
-1.4136e-01, 9.1187e-02, -6.7627e-01, -3.2056e-01, 5.1562e-01],
[ 1.1250e+00, 7.3730e-01, -3.8281e-01, 2.8271e-01, -6.7725e-01,
-2.7847e-03, 7.8125e-01, 5.4395e-01, -3.7378e-01, 1.5308e-01,
9.9414e-01, -2.6169e-02, -2.7026e-01, 3.6530e-02, -1.5979e-01,
1.3416e-01, 3.3340e-03, -3.6353e-01, -8.9172e-02, 6.4893e-01],
[ 5.4102e-01, 8.0420e-01, -1.0699e-01, 7.5635e-01, -8.3936e-01,
1.2329e-02, 1.0488e+00, 1.9812e-01, -2.9980e-01, -3.6865e-01,
6.7090e-01, 2.2644e-01, -8.3496e-01, -3.3813e-01, -2.3657e-01,
3.9209e-01, -1.5930e-01, -5.8594e-01, -3.3508e-02, -4.1107e-02],
[ 3.1177e-01, 2.3840e-01, -1.3757e-01, 2.5317e-01, -1.5137e+00,
-3.4155e-01, 4.4141e-01, 3.4277e-01, -9.0479e-01, -8.7744e-01,
-1.6510e-02, 1.2158e-01, -9.4238e-01, -4.4507e-01, -5.7959e-01,
5.5566e-01, -1.9275e-01, -4.9121e-01, -6.2744e-01, -8.9502e-01],
[ 1.5173e-01, 3.0444e-01, -1.0371e+00, -8.1494e-01, -4.1284e-01,
-4.1779e-02, -4.0375e-02, -1.9336e-01, -1.0498e+00, -4.2944e-01,
3.4155e-01, -4.5380e-02, -7.1240e-01, -8.3154e-01, -1.0322e+00,
-5.4443e-01, -6.6504e-01, -1.1064e+00, -8.6426e-01, 3.1372e-01],
[ 1.8555e-01, 7.5684e-01, -1.3586e-01, -6.1913e-03, -1.2783e+00,
4.1504e-02, 7.4609e-01, -6.7932e-02, -8.3643e-01, -3.7744e-01,
2.2278e-01, 4.6777e-01, -9.6973e-01, -9.6582e-01, -8.4131e-01,
-1.9775e-01, -5.1514e-01, -1.3555e+00, -9.7949e-01, -2.5439e-01],
[ 8.1787e-01, 6.9385e-01, -3.2153e-01, -4.4098e-02, -1.2910e+00,
2.0911e-01, 8.3203e-01, 7.4658e-01, -7.9150e-01, -5.2100e-01,
1.9727e-01, 7.8809e-01, -1.0508e+00, -8.1494e-01, -6.0852e-02,
3.9404e-01, -1.0864e-01, -1.3398e+00, -9.8682e-01, -5.5389e-02],
[ 2.0813e-01, 7.2510e-01, 1.7261e-01, -1.6675e-01, -9.7119e-01,
3.8721e-01, 7.7930e-01, 2.3608e-01, -3.4277e-01, -1.7664e-01,
3.4229e-01, 5.3906e-01, -9.1406e-01, -7.5195e-01, -4.4482e-01,
1.5552e-01, 2.7191e-02, -1.3760e+00, -7.4463e-01, 1.5039e-01],
[ 5.0830e-01, 7.3877e-01, -2.1704e-01, 5.9204e-03, -9.9121e-01,
1.7090e-01, 6.7480e-01, 7.4268e-01, -5.2051e-01, -6.5430e-02,
6.4307e-01, 6.5820e-01, -5.2832e-01, -3.9136e-01, -5.2148e-01,
7.5317e-02, -2.0007e-01, -7.1338e-01, -4.8413e-01, 2.0520e-01],
[ 6.6699e-01, 1.0410e+00, -1.6663e-01, -3.1281e-02, -7.9102e-01,
-1.9852e-02, 2.7417e-01, 8.4619e-01, -4.1162e-01, 7.3929e-03,
6.8652e-01, 6.9092e-01, -9.8145e-01, -1.1371e-01, 3.2446e-01,
5.7812e-01, -1.2598e-01, -9.0479e-01, -5.6641e-01, 2.2363e-01]],
dtype=float16)
Plotting logits#
You can plot the logits by position:
[ ]:
import matplotlib.pyplot as plt
import scipy
import numpy as np
alphabet = model.metadata.output_tokens
_, ax = plt.subplots(figsize=(20, 4))
ax.imshow(scipy.special.log_softmax(logits, axis=-1).T)
_ = ax.set_xlabel('Position')
_ = ax.set_ylabel('Amino Acid')
_ = ax.set_yticks(np.arange(len(alphabet)), alphabet)
Next steps#
For more information, visit the Embeddings API reference.