Pylerno
English

HTTP and JSON

Separate network transport from response validation and keep examples reproducible.

On this page

The concept

An HTTP response has transport behavior, a status, headers, and a body. A successful connection does not prove that the body has the structure your application needs. decode_payload is a pure parsing boundary for an expected items list, so it can be tested without network access.

fetch_json demonstrates a real standard-library request function but is not called by the default demo. It sets a timeout and reads at most one byte beyond a fixed size limit to detect oversize responses. Callers of a real integration must handle HTTP, connection, and timeout failures deliberately. Trust and validate the destination rather than accepting arbitrary URLs blindly.

Worked example

Extract the example files. From the python_from_zero directory, run the following command in a terminal. On Windows, use py in place of python3.

sh
python3 08_professional_workflow/62_http_and_json/lesson.py
python
import json
from urllib.request import Request, urlopen


def decode_payload(payload):
    data = json.loads(payload.decode("utf-8"))
    if not isinstance(data, dict) or not isinstance(data.get("items"), list):
        raise ValueError("response must contain an items list")
    return data["items"]


def fetch_json(url, *, max_bytes=100000):
    request = Request(url, headers={"Accept": "application/json"})
    with urlopen(request, timeout=5) as response:
        payload = response.read(max_bytes + 1)
    if len(payload) > max_bytes:
        raise ValueError("response exceeds size limit")
    return decode_payload(payload)


if __name__ == "__main__":
    print(decode_payload(b'{"items": ["book", "pen"]}'))

Trace the result

  1. decode_payload decodes UTF-8, parses JSON, and checks the expected top-level structure.
  2. fetch_json limits waiting time and response size before delegating to the parser.
  3. The guarded demonstration uses prepared bytes only, so running the lesson sends no network request.
text
['book', 'pen']
Line-by-line walkthrough

Line 1: import json

Load json (JSON serialization and parsing). Importing makes these names available; it is not a call to every function in the module.

Line 2: from urllib.request import Request, urlopen

Import Request, urlopen from urllib.request. This supplies HTTP request construction and transport. The caller can now use the imported names directly.

Lines 5–9: def decode_payload(payload):

Define the function decode_payload with input names payload. Definition stores the operation rather than executing its body. An ordinary call executes a normal function; a coroutine runs when awaited/scheduled, and a generator advances when consumed. Decorators can provide a context-manager or wrapper interface. Indented statements below belong to this operation.

Line 6: data = json.loads(payload.decode("utf-8"))

Evaluate json.loads(payload.decode('utf-8')), then bind or update data with the result. The call finishes before its return value is bound; a failed call does not produce a successful assignment. Assignment itself does not imply a deep copy.

Lines 7–8: if not isinstance(data, dict) or not isinstance(data.get("items"), list):

Evaluate not isinstance(data, dict) or not isinstance(data.get('items'), list). Execute the indented body if it is truthy; otherwise follow the alternative branch, if present. Branch order and indentation are part of the behavior.

Line 8: raise ValueError("response must contain an items list")

Raise ValueError('response must contain an items list'). Normal execution stops here unless an enclosing handler catches it. A from cause, when present, preserves the reason for a translated error.

Line 9: return data["items"]

Evaluate data['items'] and return that value to the caller. This ends the current function call; subsequent statements on this execution path do not run.

Lines 12–18: def fetch_json(url, *, max_bytes=100000):

Define the function fetch_json with input names url, max_bytes. Definition stores the operation rather than executing its body. An ordinary call executes a normal function; a coroutine runs when awaited/scheduled, and a generator advances when consumed. Decorators can provide a context-manager or wrapper interface. Indented statements below belong to this operation.

Line 13: request = Request(url, headers={"Accept": "application/json"})

Evaluate Request(url, headers={'Accept': 'application/json'}), then bind or update request with the result. The call finishes before its return value is bound; a failed call does not produce a successful assignment. Assignment itself does not imply a deep copy.

Lines 14–15: with urlopen(request, timeout=5) as response:

Enter the listed context manager(s), run the indented block, and leave each context afterward. The manager defines the cleanup/commit behavior; a database transaction and a file lifetime are not identical contracts.

Line 15: payload = response.read(max_bytes + 1)

Evaluate response.read(max_bytes + 1), then bind or update payload with the result. The call finishes before its return value is bound; a failed call does not produce a successful assignment. Assignment itself does not imply a deep copy.

Lines 16–17: if len(payload) > max_bytes:

Evaluate len(payload) > max_bytes. Execute the indented body if it is truthy; otherwise follow the alternative branch, if present. Branch order and indentation are part of the behavior.

Line 17: raise ValueError("response exceeds size limit")

Raise ValueError('response exceeds size limit'). Normal execution stops here unless an enclosing handler catches it. A from cause, when present, preserves the reason for a translated error.

Line 18: return decode_payload(payload)

Evaluate decode_payload(payload) and return that value to the caller. This ends the current function call; subsequent statements on this execution path do not run.

Lines 21–22: if __name__ == "__main__":

Enter the program entry point only during direct execution. Importing this module still creates definitions but does not run this guarded workflow.

Line 22: print(decode_payload(b'{"items": ["book", "pen"]}'))

Evaluate the print arguments from the current state and display them. sep controls separators and end controls the ending when those options are provided. Displaying a value does not return that value from the surrounding function.

Common mistake

JSON syntax validity is not schema validity. Do not hide connection failures by returning an indistinguishable empty result.

Check your understanding

Which HTTP status means the requested resource was not found?

Choose one answer

Search documentation

Share this page