home > research > openai-models (2024-10-30)

Navigating the OpenAI API Key and Models

Navigating the OpenAI API Key and Models

Table of Contents

OpenAI API

You can get your API key from https://beta.openai.com/account/api-keys

You can set it in your shell environment: 

    export OPENAI_API_KEY=sk-...
#+end_example

#+begin_src sh :results raw
echo "API Key starts with: $(echo $OPENAI_API_KEY | cut -c1-7)..."
echo "API Key length: ${#OPENAI_API_KEY}"

API Key starts with: sk-Q27r… API Key length: 51

Models

Shell

    curl -s -X 'GET' \
         'https://api.openai.com/v1/models' \
         -H 'accept: application/json' \
         -H "Authorization: Bearer $OPENAI_API_KEY" \
        | jq -r '.data | sort_by(-.created) | map([.id, .object, .created, .owned_by] | @tsv) | .[]'

Emacs Lisp

  (require 'json)
  (require 'url)

  (let* ((url-request-method "GET")
         (url-request-extra-headers
          `(("Authorization" . ,(concat "Bearer " openai-api-key))
            ("Accept" . "application/json")))
         (url "https://api.openai.com/v1/models")
         (buffer (url-retrieve-synchronously url))
         (response-body (with-current-buffer buffer
                          (goto-char (point-min))
                          (re-search-forward "\n\n")
                          (buffer-substring-no-properties (point) (point-max))))
         (json-object-type 'hash-table)
         (json-array-type 'list)
         (json-key-type 'string)
         (data (gethash "data" (json-read-from-string response-body)))
         (sorted-data (sort data (lambda (a b) 
                                   (> (gethash "created" a) 
                                      (gethash "created" b))))))
    (with-output-to-string
      (princ "id\tobject\tcreated\towned_by\n")
      (dolist (item sorted-data)
        (princ (format "%s\t%s\t%s\t%s\n"
                       (gethash "id" item)
                       (gethash "object" item)
                       (gethash "created" item)
                       (gethash "owned_by" item))))))

Rest

GET https://api.openai.com/v1/models
Accept: application/json
Authorization: Bearer {{{api-key}}}

GET https://api.openai.com/v1/models
Accept: application/json
Authorization: Bearer :api-key

AI Model Outputs

Create Output Directory

mkdir -p ai_model_outputs
echo "JSON outputs will be saved in the 'ai_model_outputs' directory."

JSON outputs will be saved in the 'ai_model_outputs' directory.

OpenAI Models

echo "OpenAI API Key starts with: $(echo $OPENAI_API_KEY | cut -c1-7)..."
echo "OpenAI API Key length: ${#OPENAI_API_KEY}"

echo "OpenAI Models:"
curl -s -H "Authorization: Bearer $OPENAI_API_KEY" \
     -H "Content-Type: application/json" \
     https://api.openai.com/v1/models | 
     tee ai_model_outputs/openai_models.json | jq '.data[].id' | head

echo "Model structure (keys of the first model):"
jq -r '.[0] | keys[]' ai_model_outputs/huggingface_models.json


OpenAI API Key starts with: sk-Q27r...
OpenAI API Key length: 51
OpenAI Models:
"gpt-4-turbo"
"gpt-4-turbo-2024-04-09"
"tts-1"
"tts-1-1106"
"dall-e-2"
"whisper-1"
"gpt-3.5-turbo-instruct"
"chatgpt-4o-latest"
"gpt-4o-mini"
"gpt-4o-2024-05-13"
Model structure (keys of the first model):
_id
createdAt
downloads
id
library_name
likes
modelId
pipeline_tag
private
tags

Anthropic Models

echo "Anthropic API Key starts with: $(echo $ANTHROPIC_API_KEY | cut -c1-7)..."
echo "Anthropic API Key length: ${#ANTHROPIC_API_KEY}"

echo "Anthropic Models:"
curl -s -H "x-api-key: $ANTHROPIC_API_KEY" \
     -H "Content-Type: application/json" \
     https://api.anthropic.com/v1/models | 
     tee ai_model_outputs/anthropic_models.json | jq '.models[].name'

Anthropic API Key starts with: sk-ant-...
Anthropic API Key length: 108
Anthropic Models:

Google Gemini Models

echo "Google API Key starts with: $(echo $GOOGLE_API_KEY | cut -c1-7)..."
echo "Google API Key length: ${#GOOGLE_API_KEY}"

echo "Google Gemini Models:"
curl -s -H "Authorization: Bearer $GOOGLE_API_KEY" \
     -H "Content-Type: application/json" \
     "https://generativelanguage.googleapis.com/v1beta/models?key=$GOOGLE_API_KEY" | 
     tee ai_model_outputs/google_gemini_models.json | jq '.models[].name'

Google API Key starts with: ...
Google API Key length: 0
Google Gemini Models:

Phind Models

echo "Phind Models:"
echo "Please check https://www.phind.com/api for the latest information on available models." | 
tee ai_model_outputs/phind_info.txt

Phind Models:
Please check https://www.phind.com/api for the latest information on available models.

Meta AI Models

echo "Meta AI Models:"
echo "LLaMA models are typically accessed through third-party providers or self-hosted solutions." | 
tee ai_model_outputs/meta_ai_info.txt

Meta AI Models:
LLaMA models are typically accessed through third-party providers or self-hosted solutions.

Cohere Models

echo "Cohere API Key starts with: $(echo $COHERE_API_KEY | cut -c1-7)..."
echo "Cohere API Key length: ${#COHERE_API_KEY}"

echo "Cohere Models:"
curl -s -H "Authorization: Bearer $COHERE_API_KEY" \
     -H "Content-Type: application/json" \
     https://api.cohere.ai/v1/models | 
     tee ai_model_outputs/cohere_models.json | jq '.models[].name'

Cohere API Key starts with: ...
Cohere API Key length: 0
Cohere Models:

AI21 Labs Models

echo "AI21 API Key starts with: $(echo $AI21_API_KEY | cut -c1-7)..."
echo "AI21 API Key length: ${#AI21_API_KEY}"

echo "AI21 Labs Models:"
curl -s -H "Authorization: Bearer $AI21_API_KEY" \
     -H "Content-Type: application/json" \
     https://api.ai21.com/studio/v1/models | 
     tee ai_model_outputs/ai21_models.json | jq '.models[].name'

AI21 API Key starts with: ...
AI21 API Key length: 0
AI21 Labs Models:

Hugging Face Models

echo "Hugging Face API Key starts with: $(echo $HUGGINGFACE_API_KEY | cut -c1-7)..."
echo "Hugging Face API Key length: ${#HUGGINGFACE_API_KEY}"

echo "Fetching Hugging Face Models (top models by downloads)..."
curl -s -H "Authorization: Bearer $HUGGINGFACE_API_KEY" \
     "https://huggingface.co/api/models?sort=downloads&direction=-1&limit=100" \
     > ai_model_outputs/huggingface_models.json

echo "Number of models retrieved:"
jq '. | length' ai_model_outputs/huggingface_models.json

echo "Top 5 model names:"
jq -r '.[0:5][].modelId' ai_model_outputs/huggingface_models.json

echo "Model structure (keys of the first model):"
jq -r '.[0] | keys[]' ai_model_outputs/huggingface_models.json

Hugging Face API Key starts with: ...
Hugging Face API Key length: 0
Fetching Hugging Face Models (top models by downloads)...
Number of models retrieved:
100
Top 5 model names:
BAAI/bge-base-en-v1.5
sentence-transformers/all-mpnet-base-v2
google-bert/bert-base-uncased
nesaorg/benchmark_v0
sentence-transformers/all-MiniLM-L6-v2
Model structure (keys of the first model):
_id
createdAt
downloads
id
library_name
likes
modelId
pipeline_tag
private
tags

Author: Jason Walsh

j@wal.sh

Last Updated: 2024-10-30 16:43:54