OCaml Support - atdgen¶
Tutorial¶
What is atdgen?¶
Atdgen is a tool that derives OCaml boilerplate code from type definitions. Currently it provides support for:
What are the advantages of atdgen?¶
Atdgen has a number of advantages over its predecessor json-static which was based on Camlp4:
- produces explicit interfaces which describe what is available to the user (.mli files).
- produces readable OCaml code that can be easily reviewed (.ml files).
- produces fast code, 3x faster than json-static.
- runs fast, keeping build times low.
- same ATD definitions can be used to generate code other than OCaml. See for instance atdj which generates Java classes for JSON IO. Auto-generating GUI widgets from type definitions is another popular use of annotated type definitions. The implementation of such code generators is facilitated by the atd library.
Prerequisites¶
This tutorial assumes that you are using atdgen version 1.5.0 or above. The following command tells you which version you are using:
$ atdgen -version
1.5.0
The recommended way of installing atdgen and all its dependencies is with opam:
$ opam install atdgen
Getting started¶
From now on we assume that atdgen 1.5.0 or above is installed properly.
$ atdgen -version
1.5.0
Type definitions are placed in a .atd file (hello.atd):
type date = {
year : int;
month : int;
day : int;
}
Our handwritten OCaml program is hello.ml
:
open Hello_t
let () =
let date = { year = 1970; month = 1; day = 1 } in
print_endline (Hello_j.string_of_date date)
We produce OCaml code from the type definitions using atdgen
:
$ atdgen -t hello.atd # produces OCaml type definitions
$ atdgen -j hello.atd # produces OCaml code dealing with JSON
We now have _t and _j files produced by atdgen -t and atdgen -j respectively:
$ ls
hello.atd hello.ml hello_j.ml hello_j.mli hello_t.ml hello_t.mli
We compile all .mli and .ml files:
$ ocamlfind ocamlc -c hello_t.mli -package atdgen
$ ocamlfind ocamlc -c hello_j.mli -package atdgen
$ ocamlfind ocamlopt -c hello_t.ml -package atdgen
$ ocamlfind ocamlopt -c hello_j.ml -package atdgen
$ ocamlfind ocamlopt -c hello.ml -package atdgen
$ ocamlfind ocamlopt -o hello hello_t.cmx hello_j.cmx hello.cmx -package atdgen -linkpkg
And finally we run our hello program:
$ ./hello
{"year":1970,"month":1,"day":1}
Inspecting and pretty-printing JSON¶
Input JSON data:
$ cat single.json
[1234,"abcde",{"start_date":{"year":1970,"month":1,"day":1},
"end_date":{"year":1980,"month":1,"day":1}}]
Pretty-printed JSON can be produced with the ydump
command:
$ ydump single.json
[
1234,
"abcde",
{
"start_date": { "year": 1970, "month": 1, "day": 1 },
"end_date": { "year": 1980, "month": 1, "day": 1 }
}
]
Multiple JSON objects separated by whitespace, typically one JSON object per line, can also be pretty-printed with ydump. Input:
$ cat stream.json
[1234,"abcde",{"start_date":{"year":1970,"month":1,"day":1},
"end_date":{"year":1980,"month":1,"day":1}}]
[1,"a",{}]
In this case the -s option is required:
$ ydump -s stream.json
[
1234,
"abcde",
{
"start_date": { "year": 1970, "month": 1, "day": 1 },
"end_date": { "year": 1980, "month": 1, "day": 1 }
}
]
[ 1, "a", {} ]
From an OCaml program, pretty-printing can be done with Yojson.Safe.prettify which has the following signature:
val prettify : string -> string
We wrote a tiny program that simply calls the prettify function on some predefined JSON data (file prettify.ml):
let json =
"[1234,\"abcde\",{\"start_date\":{\"year\":1970,\"month\":1,\"day\":1},
\"end_date\":{\"year\":1980,\"month\":1,\"day\":1}}]"
let () = print_endline (Yojson.Safe.prettify json)
We now compile and run prettify.ml:
$ ocamlfind ocamlopt -o prettify prettify.ml -package atdgen -linkpkg
$ ./prettify
[
1234,
"abcde",
{
"start_date": { "year": 1970, "month": 1, "day": 1 },
"end_date": { "year": 1980, "month": 1, "day": 1 }
}
]
Inspecting biniou data¶
Biniou is a binary format that can be displayed as text using a generic command
called bdump
. The only practical difficulty is to recover the original field
names and variant names which are stored as 31-bit hashes. Unhashing them is
done by consulting a dictionary (list of words) maintained by the user.
Let’s first produce a sample data file tree.dat
containing the
biniou representation of a binary tree. In the same program
we will also demonstrate how to render biniou data into text from an
OCaml program.
Here is the ATD file defining our tree type (file tree.atd
):
type tree = [
| Empty
| Node of (tree * int * tree)
]
This is our OCaml program (file tree.ml
):
open Printf
(* sample value *)
let tree : Tree_t.tree =
`Node (
`Node (`Empty, 1, `Empty),
2,
`Node (
`Node (`Empty, 3, `Empty),
4,
`Node (`Empty, 5, `Empty)
)
)
let () =
(* write sample value to file *)
let fname = "tree.dat" in
Atdgen_runtime.Util.Biniou.to_file Tree_b.write_tree fname tree;
(* write sample value to string *)
let s = Tree_b.string_of_tree tree in
printf "raw value (saved as %s):\n%S\n" fname s;
printf "length: %i\n" (String.length s);
printf "pretty-printed value (without dictionary):\n";
print_endline (Bi_io.view s);
printf "pretty-printed value (with dictionary):\n";
let unhash = Bi_io.make_unhash ["Empty"; "Node"; "foo"; "bar" ] in
print_endline (Bi_io.view ~unhash s)
Compilation:
$ atdgen -t tree.atd
$ atdgen -b tree.atd
$ ocamlfind ocamlopt -o tree \
tree_t.mli tree_t.ml tree_b.mli tree_b.ml tree.ml \
-package atdgen -linkpkg
Running the program:
$ ./tree
raw value (saved as tree.dat):
"\023\179\2276\"\020\003\023\179\2276\"\020\003\023\003\007\170m\017\002\023\003\007\170m\017\004\023\179\2276\"\020\003\023\179\2276\"\020\003\023\003\007\170m\017\006\023\003\007\170m\017\b\023\179\2276\"\020\003\023\003\007\170m\017\n\023\003\007\170m"
length: 75
pretty-printed value (without dictionary):
<#33e33622:
(<#33e33622: (<#0307aa6d>, 1, <#0307aa6d>)>,
2,
<#33e33622:
(<#33e33622: (<#0307aa6d>, 3, <#0307aa6d>)>,
4,
<#33e33622: (<#0307aa6d>, 5, <#0307aa6d>)>)>)>
pretty-printed value (with dictionary):
<"Node":
(<"Node": (<"Empty">, 1, <"Empty">)>,
2,
<"Node":
(<"Node": (<"Empty">, 3, <"Empty">)>,
4,
<"Node": (<"Empty">, 5, <"Empty">)>)>)>
Now let’s see how to pretty-print any biniou data from the command line. Our sample data are now in file tree.dat:
$ ls -l tree.dat
-rw-r--r-- 1 martin martin 75 Apr 17 01:46 tree.dat
We use the command bdump
to render our sample biniou data as text:
$ bdump tree.dat
<#33e33622:
(<#33e33622: (<#0307aa6d>, 1, <#0307aa6d>)>,
2,
<#33e33622:
(<#33e33622: (<#0307aa6d>, 3, <#0307aa6d>)>,
4,
<#33e33622: (<#0307aa6d>, 5, <#0307aa6d>)>)>)>
We got hashes for the variant names Empty
and Node
. Let’s add them to the
dictionary:
$ bdump -w Empty,Node tree.dat
<"Node":
(<"Node": (<"Empty">, 1, <"Empty">)>,
2,
<"Node":
(<"Node": (<"Empty">, 3, <"Empty">)>,
4,
<"Node": (<"Empty">, 5, <"Empty">)>)>)>
bdump
remembers the dictionary so we don’t have to pass the -w
option
anymore (for this user on this machine). The following now works:
$ bdump tree.dat
<"Node":
(<"Node": (<"Empty">, 1, <"Empty">)>,
2,
<"Node":
(<"Node": (<"Empty">, 3, <"Empty">)>,
4,
<"Node": (<"Empty">, 5, <"Empty">)>)>)>
Optional fields and default values¶
Although OCaml records do not support optional fields, both the JSON and biniou formats make it possible to omit certain fields on a per-record basis.
For example the JSON record { “x”: 0, “y”: 0 } can be more compactly written as {} if the reader knows the default values for the missing fields x and y. Here is the corresponding type definition:
type vector_v1 = { ~x: int; ~y: int }
~x
means that field x
supports a default value. Since we do not specify
the default value ourselves, the built-in default is used, which is 0.
If we want the default to be something else than 0, we just have to specify it as follows:
type vector_v2 = {
~x <ocaml default="1">: int; (* default x is 1 *)
~y: int; (* default y is 0 *)
}
It is also possible to specify optional fields without a default value. For
example, let’s add an optional z
field:
type vector_v3 = {
~x: int;
~y: int;
?z: int option;
}
The following two examples are valid JSON representations of data of type
vector_v3
:
{ "x": 2, "y": 2, "z": 3 } // OCaml: { x = 2; y = 2; z = Some 3 }
{ "x": 2, "y": 2 } // OCaml: { x = 2; y = 2; z = None }
By default, JSON fields whose value is null
are treated
as missing fields. The following two JSON objects are therefore equivalent:
{ "x": 2, "y": 2, "z": null }
{ "x": 2, "y": 2 }
Note also the difference between ?z: int option
and ~z: int option
:
type vector_v4 = {
~x: int;
~y: int;
~z: int option; (* no unwrapping of the JSON field value! *)
}
Here are valid values of type vector_v4
, showing that it is usually not what
is intended:
{ "x": 2, "y": 2, "z": [ "Some", 3 ] }
{ "x": 2, "y": 2, "z": "None" }
{ "x": 2, "y": 2 }
Smooth protocol upgrades¶
Problem: you have a production system that uses a specific JSON or biniou format. It may be data files or a client-server pair. You now want to add a field to a record type or to add a case to a variant type.
Both JSON and biniou allow extra record fields. If the consumer does not know how to deal with the extra field, the default behavior is to happily ignore it.
Adding or removing an optional record field¶
type t = {
x: int;
y: int;
}
Same .atd
source file, edited:
type t = {
x: int;
y: int;
~z: int; (* new field *)
}
- Upgrade producers and consumers in any order
- Converting old data is not required nor useful
Adding a required record field¶
type t = {
x: int;
y: int;
}
Same .atd
source file, edited:
type t = {
x: int;
y: int;
z: int; (* new field *)
}
- Upgrade all producers before the consumers
- Converting old data requires special-purpose hand-written code
Removing a required record field¶
- Upgrade all consumers before the producers
- Converting old data is not required but may save some storage space (just read and re^write each record using the new type)
Adding a variant case¶
type t = [ A | B ]
Same .atd
source file, edited:
type t = [ A | B | C ]
- Upgrade all consumers before the producers
- Converting old data is not required and would have no effect
Removing a variant case¶
- Upgrade all producers before the consumers
- Converting old data requires special^purpose hand^written code
Avoiding future problems¶
- In doubt, use records rather than tuples because it makes it possible to add or remove any field or to reorder them.
- Do not hesitate to create variant types with only one case or records with only one field if you think they might be extended later.
Data validation¶
Atdgen can be used to produce data validators for all types defined in an ATD file, based on user-given validators specified only for certain types. A simple example is:
type t = string <ocaml valid="fun s -> String.length s >= 8"> option
As we can see from this example, the validation function is specified using the
annotation <ocaml valid="p">
, where p
is a predicate p : t -> bool
,
returning true
when the value of type t
is valid and false
otherwise.
Calling atdgen -v
on a file containing this specification will produce
a validation function equivalent to the following implementation:
let validate_t path x =
match x with
| None -> None
| Some x ->
let msg = "Failed check by fun s -> String.length s >= 8" in
if (fun s -> String.length s >= 8) x
then None
else Some {error_path = path; error_msg = msg}
Let’s consider this particular example as an illustration of the general shape of generated validation functions.
The function takes two arguments: the first, path
, is a list indicating
where the second, x
, was encountered. As specified by our example .atd
code above, x
has type t option
.
The body of the validation function does two things:
1. it checks the value of x
against the validation function specified in our
.atd
file, namely, checking whether there is Some s
, and verifying that
s
is at least 8 characters long if so
2. in the event that the validation check fails, it constructs an appropriate
error record.
In general, generated validation functions for a type t
have a type
equivalent to validate_t : path -> t -> error option
, where the path
gives the current location in a data structure and the error
is a record of
the location of, and reason for, validation failure.
A return value of None
indicates successful validation, while Some
{error_path; error_msg}
tells us where and why validation failed.
Let’s now consider a more realistic example with complex validators defined in a
separate .ml
file. We will define a data structure representing a section of
a resume recording work experience. We will also define validation functions
that can enforce certain properties to protect against errors and junk data.
In the course of this example, we will manually create the following 3 source files:
resume.atd
: contains the type definitions with annotationsresume_util.ml
: contains our handwritten validatorsresume.ml
: is our main program that creates data and checks it using our generated validation functions.
After generating additional code with atdgen
, we will end up with the
following OCaml modules:
Resume_t
: generated intoresume_t.ml
byatdgen -t resume.atd
, this provides our OCaml type definitionsResume_util
: written manually inresume_util.ml
, this depends onResume_t
and provides validators we will use inresume.atd
Resume_v
: generated intoresume_v.ml
byatdgen -v resume.atd
, this depends onResume_util
andResume_t
and provides a validation function for each typeResume_j
: generated intoresume_j.ml
byatdgen -j resume.atd
, this provides functions to serialize and deserialize data in and out of JSON.Resume
: written manually inresume.ml
, this depends onResume_v
, andResume_t
, and makes use of the generated types and validation functions.
To begin, we specify type definitions for a data structure representing a resume
in resume.atd
:
type text = string <ocaml valid="Resume_util.validate_some_text">
type date = {
year : int;
month : int;
day : int;
} <ocaml valid="Resume_util.validate_date">
type job = {
company : text;
title : text;
start_date : date;
?end_date : date option;
} <ocaml valid="Resume_util.validate_job">
type work_experience = job list
We can now call atdgen -t resume.atd
to generate our Resume_t
module in
resume_t.ml
, providing our data types. Using these data types, we’ll define
the following handwritten validators in resume_util.ml
(note that we’ve
already referred to these validators in resume.atd
):
open Resume_t
let ascii_printable c =
let n = Char.code c in
n >= 32 && n <= 127
(*
Check that string is not empty and contains only ASCII printable
characters (for the sake of the example; we use UTF-8 these days)
*)
let validate_some_text s =
s <> "" &&
try
String.iter (fun c -> if not (ascii_printable c) then raise Exit) s;
true
with Exit ->
false
(*
Check that the combination of year, month and day exists in the
Gregorian calendar.
*)
let validate_date x =
let y = x.year in
let m = x.month in
let d = x.day in
m >= 1 && m <= 12 && d >= 1 &&
(let dmax =
match m with
2 ->
if y mod 4 = 0 && not (y mod 100 = 0) || y mod 400 = 0 then 29
else 28
| 1 | 3 | 5 | 7 | 8 | 10 | 12 -> 31
| _ -> 30
in
d <= dmax)
(* Compare dates chronologically *)
let compare_date a b =
let c = compare a.year b.year in
if c <> 0 then c
else
let c = compare a.month b.month in
if c <> 0 then c
else compare a.day b.day
(* Check that the end_date, when defined, is not earlier than the start_date *)
let validate_job x =
match x.end_date with
None -> true
| Some end_date ->
compare_date x.start_date end_date <= 0
After we call atdgen -v resume.atd
, the module Resume_v
will be
generated in resume_v.ml
, providing the function
validate_work_experience
. We can then use this function, along with the
generated Resume_j
in the following program written in resume.ml
:
let check_experience x =
let is_valid = match Resume_v.validate_work_experience [] x with
| None -> false
| _ -> true
in
Printf.printf "%s:\n%s\n"
(if is_valid then "VALID" else "INVALID")
(Yojson.Safe.prettify (Resume_j.string_of_work_experience x))
let () =
(* one valid date *)
let valid = { Resume_t.year = 2000; month = 2; day = 29 } in
(* one invalid date *)
let invalid = { Resume_t.year = 1900; month = 0; day = 0 } in
(* two more valid dates, created with Resume_v.create_date *)
let date1 = { Resume_t.year = 2005; month = 8; day = 1 } in
let date2 = { Resume_t.year = 2006; month = 3; day = 22 } in
let job = {
Resume_t.company = "Acme Corp.";
title = "Tester";
start_date = date1;
end_date = Some date2;
}
in
let valid_job = { job with Resume_t.start_date = valid } in
let invalid_job = { job with Resume_t.end_date = Some invalid } in
let valid_experience = [ job; valid_job ] in
let invalid_experience = [ job; invalid_job ] in
check_experience valid_experience;
check_experience invalid_experience
Output:
VALID:
[
{
"company": "Acme Corp.",
"title": "Tester",
"start_date": { "year": 2005, "month": 8, "day": 1 },
"end_date": { "year": 2006, "month": 3, "day": 22 }
},
{
"company": "Acme Corp.",
"title": "Tester",
"start_date": { "year": 2000, "month": 2, "day": 29 },
"end_date": { "year": 2006, "month": 3, "day": 22 }
}
]
INVALID:
[
{
"company": "Acme Corp.",
"title": "Tester",
"start_date": { "year": 2005, "month": 8, "day": 1 },
"end_date": { "year": 2006, "month": 3, "day": 22 }
},
{
"company": "Acme Corp.",
"title": "Tester",
"start_date": { "year": 2005, "month": 8, "day": 1 },
"end_date": { "year": 1900, "month": 0, "day": 0 }
}
Modularity: referring to type definitions from another ATD file¶
It is possible to define types that depend on types defined in other .atd
files. The example below is self-explanatory.
part1.atd
:
type t = { x : int; y : int }
part2.atd
:
type t1 <ocaml from="Part1" t="t"> = abstract
(*
Imports type t defined in file part1.atd.
The local name is t1. Because the local name (t1) is different from the
original name (t), we must specify the original name using t=.
*)
type t2 = t1 list
part3.atd
:
type t2 <ocaml from="Part2"> = abstract
type t3 = {
name : string;
?data : t2 option;
}
main.ml
:
let v = {
Part3_t.name = "foo";
data = Some [
{ Part1_t.x = 1; y = 2 };
{ Part1_t.x = 3; y = 4 };
]
}
let () =
Atdgen_runtime.Util.Json.to_channel Part3_j.write_t3 stdout v;
print_newline ()
Output:
{"name":"foo","data":[{"x":1,"y":2},{"x":3,"y":4}]}
Managing JSON configuration files¶
JSON makes a good format for configuration files because it is human-readable, easy to modify programmatically and widespread. Here is an example of how to use atdgen to manage config files.
- Specifying defaults is done in the .atd file. See section [Optional fields and default values] for details on how to do that.
- Auto-generating a template config file with default values: a sample value in the OCaml world needs to be created but only fields without default need to be specified.
- Describing the format is achieved by embedding the .atd type definitions in the OCaml program and printing it out on request.
- Loading a config file and reporting illegal fields is achieved using the
JSON deserializers produced by
atdgen -j
. Option-j-strict-fields
ensures the misspelled field names are not ignored but reported as errors. - Reindenting a config file is achieved by the pretty-printing function
Yojson.Safe.prettify
that takes a JSON string and returns an equivalent JSON string. - Showing implicit (default) settings is achieved by passing the
-j-defaults
option toatdgen
. The OCaml config data is then serialized into JSON containing all fields, including those whose value is the default.
The example uses the following type definitions:
type config = {
title : string;
?description : string option;
~timeout <ocaml default="10"> : int;
~credentials : param list
<ocaml valid="fun l ->
l <> [] || failwith \"missing credentials\"">;
}
type param = {
name : string
<ocaml valid="fun s -> s <> \"\"">;
key : string
<ocaml valid="fun s -> String.length s = 16">;
}
Our program will perform the following actions:
$ ./config -template
{
"title": "",
"timeout": 10,
"credentials": [ { "name": "foo", "key": "0123456789abcdef" } ]
}
$ ./config -format
type config = {
title : string;
?description : string option;
~timeout <ocaml default="10"> : int;
~credentials : param list
<ocaml valid="fun l ->
l <> [] || failwith \"missing credentials\"">;
}
type param = {
name : string
<ocaml valid="fun s -> s <> \"\"">;
key : string
<ocaml valid="fun s -> String.length s = 16">;
}
$ cat sample-config.json
{
"title": "Example",
"credentials": [
{
"name": "joeuser",
"key": "db7c0877bdef3016"
},
{
"name": "tester",
"key": "09871ff387ac2b10"
}
]
}
$ ./config -validate sample-config.json
{
"title": "Example",
"timeout": 10,
"credentials": [
{ "name": "joeuser", "key": "db7c0877bdef3016" },
{ "name": "tester", "key": "09871ff387ac2b10" }
]
}
This is our demo.sh script that builds and runs our example program called config:
#! /bin/sh -e
set -x
# Embed the contents of the .atd file into our OCaml program
echo 'let contents = "\' > config_atd.ml
sed -e 's/\([\\"]\)/\\\1/g' config.atd >> config_atd.ml
echo '"' >> config_atd.ml
# Derive OCaml type definitions from .atd file
atdgen -t config.atd
# Derive JSON-related functions from .atd file
atdgen -j -j-defaults -j-strict-fields config.atd
# Derive validator from .atd file
atdgen -v config.atd
# Compile the OCaml program
ocamlfind ocamlopt -o config \
config_t.mli config_t.ml config_j.mli config_j.ml config_v.mli config_v.ml \
config_atd.ml config.ml -package atdgen -linkpkg
# Output a sample config
./config -template
# Print the original type definitions
./config -format
# Fail to validate an invalid config file
./config -validate bad-config1.json || :
# Fail to validate another invalid config file (using custom validators)
./config -validate bad-config3.json || :
# Validate, inject missing defaults and pretty-print
./config -validate sample-config.json
This is the hand-written OCaml program. It can be used as a start
point for a real-world program using a JSON config file:
open Printf
let param_template =
(* Sample item used to populate the template config file *)
{
Config_v.name = "foo";
key = "0123456789abcdef"
}
let config_template =
(*
Records can be conveniently created using functions generated by
"atdgen -v".
Here we use Config_v.create_config to create a record of type
Config_t.config. The big advantage over creating the record
directly using the record notation {...} is that we don't have to
specify default values (such as timeout in this example).
*)
Config_v.create_config ~title:"" ~credentials: [param_template] ()
let make_json_template () =
(* Thanks to the -j-defaults flag passed to atdgen, even default
fields will be printed out *)
let compact_json = Config_j.string_of_config config_template in
Yojson.Safe.prettify compact_json
let print_template () =
print_endline (make_json_template ())
let print_format () =
print_string Config_atd.contents
let validate fname =
let x =
try
(* Read config data structure from JSON file *)
let x = Atdgen_runtime.Util.Json.from_file Config_j.read_config fname in
(* Call the validators specified by <ocaml valid=...> *)
if not (Config_v.validate_config x) then
failwith "Some fields are invalid"
else
x
with e ->
(* Print decent error message and exit *)
let msg =
match e with
Failure s
| Yojson.Json_error s -> s
| e -> Printexc.to_string e
in
eprintf "Error: %s\n%!" msg;
exit 1
in
(* Convert config to compact JSON and pretty-print it.
~std:true means that the output will not use extended syntax for
variants and tuples but only standard JSON. *)
let json = Yojson.Safe.prettify ~std:true (Config_j.string_of_config x) in
print_endline json
type action = Template | Format | Validate of string
let main () =
let action = ref Template in
let options = [
"-template", Arg.Unit (fun () -> action := Template),
"
prints a sample configuration file";
"-format", Arg.Unit (fun () -> action := Format),
"
prints the format specification of the config files (atd format)";
"-validate", Arg.String (fun s -> action := Validate s),
"<CONFIG FILE>
reads a config file, validates it, adds default values
and prints the config nicely to stdout";
]
in
let usage_msg = sprintf "\
Usage: %s [-template|-format|-validate ...]
Demonstration of how to manage JSON configuration files with atdgen.
"
Sys.argv.(0)
in
let anon_fun s = eprintf "Invalid command parameter %S\n%!" s; exit 1 in
Arg.parse options anon_fun usage_msg;
match !action with
Template -> print_template ()
| Format -> print_format ()
| Validate s -> validate s
let () = main ()
The full source code for this section with examples can be inspected and downloaded here.
Integration with ocamldoc¶
Ocamldoc is a tool that comes with the core OCaml distribution. It uses comments within (** and *) to produce hyperlinked documentation (HTML) of module signatures.
Atdgen can produce .mli files with comments in the syntax supported by ocamldoc but regular ATD comments within (* and *) are always discarded by atdgen. Instead, <doc text=”…”> must be used and placed after the element they describe. The contents of the text field must be UTF8-encoded.
type point = {
x : float;
y : float;
~z
<doc text="Optional depth, its default value is {{0.0}}.">
: float;
}
<doc text="Point with optional 3rd dimension.
OCaml example:
{{{
let p =
{ x = 0.5; y = 1.0; z = 0. }
}}}
">
is converted into the following .mli file with ocamldoc-compatible comments:
(**
Point with optional 3rd dimension.
OCaml example:
{v
let p =
\{ x = 0.5; y = 1.0; z = 0. \}
v}
*)
type point = {
x: float;
y: float;
z: float (** Optional depth, its default value is [0.0]. *)
}
The only two forms of markup supported by <doc text="...">
are {{
…
}}
for inline code and {{{
… }}}
for a block of preformatted code.
Integration with build systems¶
OMake¶
We provide an Atdgen plugin for OMake. It simplifies the compilation rules to a minimum.
The plugin consists of a self-documented file to copy into a project’s root. The
following is a sample OMakefile
for a project using JSON and five source
files (foo.atd
, foo.ml
, bar.atd
, bar.ml
and main.ml
):
# require file Atdgen.om
include Atdgen
# OCaml modules we want to build
OCAMLFILES = foo_t foo_j foo bar_t bar_j bar main
Atdgen(foo bar, -j-std)
OCamlProgram(foobar, $(OCAMLFILES))
.DEFAULT: foobar.opt
.PHONY: clean
clean:
rm -f *.cm[ioxa] *.cmx[as] *.[oa] *.opt *.run *~
rm -f $(ATDGEN_OUTFILES)
Running omake
builds the native code executable foobar.opt
.
omake clean
removes all the products of compilation including the .mli
and .ml
produced by atdgen
.
GNU Make¶
We provide Atdgen.mk, a generic
makefile that defines the dependencies and rules for generating OCaml .mli
and
.ml
files from .atd
files containing type definitions. The Atdgen.mk
file
contains its own documentation.
Here is a sample Makefile that takes advantage of OCamlMakefile:
.PHONY: default
default: opt
ATDGEN_SOURCES = foo.atd bar.atd
ATDGEN_FLAGS = -j-std
include Atdgen.mk
SOURCES = \
foo_t.mli foo_t.ml foo_j.mli foo_j.ml \
bar_t.mli bar_t.ml bar_j.mli bar_j.ml \
hello.ml
RESULT = hello
PACKS = atdgen
# "include OCamlMakefile" must come after defs for SOURCES, RESULT, PACKS, etc.
include OCamlMakefile
.PHONY: sources opt all
sources: $(SOURCES)
opt: sources
$(MAKE) native-code
all: sources
$(MAKE) byte-code
make
alone builds a native code executable from source files foo.atd
,
bar.atd
and hello.ml
. make clean
removes generated files. make
all
builds a bytecode executable.
In addition to native-code
, byte-code
and clean
, OCamlMakefile
provides a number of other targets and options which are documented in
OCamlMakefile
’s README.
Ocamlbuild¶
There is an atdgen plugin for ocamlbuild.
Dune (formerly jbuilder)¶
Dune currently needs atdgen build rules specified manually. Given an example.atd
,
this will usually look like:
(rule
(targets example_j.ml
example_j.mli)
(deps example.atd)
(action (run atdgen -j -j-std %{deps})))
(rule
(targets example_t.ml
example_t.mli)
(deps example.atd)
(action (run atdgen -t %{deps})))
You can refer to example_t.ml
and example_j.ml
as usual (by default, they
will be automatically linked into the library being built in the same directory).
You will need to write rules for each .atd file individually until
Dune supports wildcard rules.
Note that any options atdgen
supports can be included in the run atdgen
section (-open
, -deriving-conv
, etc.).
Dealing with untypable JSON¶
Sometimes we have to deal with JSON data that cannot be described using type definitions. In such case, we can represent the data as its JSON abstract syntax tree (AST), which lets the user inspect it at runtime.
Let’s consider a list of JSON objects for which we don’t know the type definitions, but somehow some other system knows how to deal with such data. Here is such data:
[
{
"label": "flower",
"value": {
"petals": [12, 45, 83.5555],
"water": "a340bcf02e"
}
},
{
"label": "flower",
"value": {
"petals": "undefined",
"fold": null,
"water": 0
}
},
{ "labels": ["fork", "scissors"],
"value": [ 8, 8 ]
}
]
Hopefully this means something for someone. We are going to assume that each
object has a value
field of an unknown type, and may have a field label
or a field labels
of type string
:
(* File untypable.atd *)
type obj_list = obj list
type obj = {
?label: string option;
?labels: string list option;
value: abstract (* requires ATD >= 2.6.0 *)
}
Until ATD 2.5, abstract
could not be used as freely and would not
stand for raw JSON by default. One had to write a dedicated type
definition as shown below:
(* File untypable.atd *)
(* deprecated since ATD 2.6 *)
type json <ocaml module="Yojson.Safe"> = abstract
(* uses type Yojson.Safe.t,
with the functions Yojson.Safe.write_json
and Yojson.Safe.read_json *)
type obj_list = obj list
type obj = {
?label: string option;
?labels: string list option;
value: json
}
It is possible to give a different name than json
to the type of the JSON
AST, but then the name of the type used in the original module must be provided
in the annotation, i.e.:
(* deprecated since ATD 2.6 *)
type raw_json <ocaml module="Yojson.Safe" t="json"> = abstract
(* uses type Yojson.Safe.t,
with the functions Yojson.Safe.write_json
and Yojson.Safe.read_json *)
type obj_list = obj list
type obj = {
?label: string option;
?labels: string list option;
value: raw_json
}
Compile either example with:
$ atdgen -t untypable.atd
$ atdgen -j -j-std untypable.atd
$ ocamlfind ocamlc -a -o untypable.cma -package atdgen \
untypable_t.mli untypable_t.ml untypable_j.mli untypable_j.ml
Test the example with your favorite OCaml toplevel (ocaml
or utop
):
# #use "topfind";;
# #require "atdgen";;
# #load "untypable.cma";;
# Atdgen_runtime.Util.Json.from_channel Untypable_j.read_obj_list stdin;;
[
{
"label": "flower",
"value": {
"petals": [12, 45, 83.5555],
"water": "a340bcf02e"
}
},
{
"label": "flower",
"value": {
"petals": "undefined",
"fold": null,
"water": 0
}
},
{ "labels": ["fork", "scissors"],
"value": [ 8, 8 ]
}
]
- : Untypable_t.obj_list =
[{Untypable_t.label = Some "flower"; labels = None;
value =
`Assoc
[("petals", `List [`Int 12; `Int 45; `Float 83.5555]);
("water", `String "a340bcf02e")]};
{Untypable_t.label = Some "flower"; labels = None;
value =
`Assoc [("petals", `String "undefined");
("fold", `Null);
("water", `Int 0)]};
{Untypable_t.label = None; labels = Some ["fork"; "scissors"];
value = `List [`Int 8; `Int 8]}]
Atdgen reference¶
Description¶
Atdgen is a command-line program that takes as input type definitions in the ATD syntax and produces OCaml code suitable for data serialization and deserialization.
Two data formats are currently supported, these are JSON and biniou, a binary format with extensibility properties similar to JSON. Atdgen-json and Atdgen-biniou will refer to Atdgen used in one context or the other.
Atdgen was designed with efficiency and durability in mind. Software authors are encouraged to use Atdgen directly and to write tools that may reuse part of Atdgen’s source code.
Atdgen uses the following packages that were developed in conjunction with Atdgen:
atd
: parser for the syntax of type definitionsbiniou
: parser and printer for biniou, a binary extensible data format`yojson
<https://github.com/ocaml-community/yojson>`__: parser and printer for JSON, a widespread text-based data format
Command-line usage¶
Command-line help¶
Call atdgen -help
for the full list of available options.
Atdgen-json example¶
$ atdgen -t example.atd
$ atdgen -j -j-std example.atd
Input file example.atd
:
type profile = {
id : string;
email : string;
~email_validated : bool;
name : string;
?real_name : string option;
~about_me : string list;
?gender : gender option;
?date_of_birth : date option;
}
type gender = [ Female | Male ]
type date = {
year : int;
month : int;
day : int;
}
is used to produce files example_t.mli
, example_t.ml
,
example_j.mli
and example_j.ml
. This is example_j.mli
:
(* Auto-generated from "example.atd" *)
type gender = Example_t.gender
type date = Example_t.date = { year: int; month: int; day: int }
type profile = Example_t.profile = {
id: string;
email: string;
email_validated: bool;
name: string;
real_name: string option;
about_me: string list;
gender: gender option;
date_of_birth: date option
}
val write_gender :
Bi_outbuf.t -> gender -> unit
(** Output a JSON value of type {!gender}. *)
val string_of_gender :
?len:int -> gender -> string
(** Serialize a value of type {!gender}
into a JSON string.
@param len specifies the initial length
of the buffer used internally.
Default: 1024. *)
val read_gender :
Yojson.Safe.lexer_state -> Lexing.lexbuf -> gender
(** Input JSON data of type {!gender}. *)
val gender_of_string :
string -> gender
(** Deserialize JSON data of type {!gender}. *)
val write_date :
Bi_outbuf.t -> date -> unit
(** Output a JSON value of type {!date}. *)
val string_of_date :
?len:int -> date -> string
(** Serialize a value of type {!date}
into a JSON string.
@param len specifies the initial length
of the buffer used internally.
Default: 1024. *)
val read_date :
Yojson.Safe.lexer_state -> Lexing.lexbuf -> date
(** Input JSON data of type {!date}. *)
val date_of_string :
string -> date
(** Deserialize JSON data of type {!date}. *)
val write_profile :
Bi_outbuf.t -> profile -> unit
(** Output a JSON value of type {!profile}. *)
val string_of_profile :
?len:int -> profile -> string
(** Serialize a value of type {!profile}
into a JSON string.
@param len specifies the initial length
of the buffer used internally.
Default: 1024. *)
val read_profile :
Yojson.Safe.lexer_state -> Lexing.lexbuf -> profile
(** Input JSON data of type {!profile}. *)
val profile_of_string :
string -> profile
(** Deserialize JSON data of type {!profile}. *)
Module Example_t
(files example_t.mli
and example_t.ml
)
contains all OCaml type definitions that can be used independently from
Biniou or JSON.
For convenience, these definitions are also made available from the
Example_j
module whose interface is shown above. Any type name,
record field name or variant constructor can be referred to using either
module. For example, the OCaml expressions
((x : Example_t.date) : Example_j.date)
and
x.Example_t.year = x.Example_j.year
are both valid.
Atdgen-biniou example¶
$ atdgen -t example.atd
$ atdgen -b example.atd
Input file example.atd
:
type profile = {
id : string;
email : string;
~email_validated : bool;
name : string;
?real_name : string option;
~about_me : string list;
?gender : gender option;
?date_of_birth : date option;
}
type gender = [ Female | Male ]
type date = {
year : int;
month : int;
day : int;
}
is used to produce files example_t.mli
, example_t.ml
,
example_b.mli
and example_b.ml
.
This is example_b.mli
:
(* Auto-generated from "example.atd" *)
type gender = Example_t.gender
type date = Example_t.date = { year: int; month: int; day: int }
type profile = Example_t.profile = {
id: string;
email: string;
email_validated: bool;
name: string;
real_name: string option;
about_me: string list;
gender: gender option;
date_of_birth: date option
}
(* Writers for type gender *)
val gender_tag : Bi_io.node_tag
(** Tag used by the writers for type {!gender}.
Readers may support more than just this tag. *)
val write_untagged_gender :
Bi_outbuf.t -> gender -> unit
(** Output an untagged biniou value of type {!gender}. *)
val write_gender :
Bi_outbuf.t -> gender -> unit
(** Output a biniou value of type {!gender}. *)
val string_of_gender :
?len:int -> gender -> string
(** Serialize a value of type {!gender} into
a biniou string. *)
(* Readers for type gender *)
val get_gender_reader :
Bi_io.node_tag -> (Bi_inbuf.t -> gender)
(** Return a function that reads an untagged
biniou value of type {!gender}. *)
val read_gender :
Bi_inbuf.t -> gender
(** Input a tagged biniou value of type {!gender}. *)
val gender_of_string :
?pos:int -> string -> gender
(** Deserialize a biniou value of type {!gender}.
@param pos specifies the position where
reading starts. Default: 0. *)
(* Writers for type date *)
val date_tag : Bi_io.node_tag
(** Tag used by the writers for type {!date}.
Readers may support more than just this tag. *)
val write_untagged_date :
Bi_outbuf.t -> date -> unit
(** Output an untagged biniou value of type {!date}. *)
val write_date :
Bi_outbuf.t -> date -> unit
(** Output a biniou value of type {!date}. *)
val string_of_date :
?len:int -> date -> string
(** Serialize a value of type {!date} into
a biniou string. *)
(* Readers for type date *)
val get_date_reader :
Bi_io.node_tag -> (Bi_inbuf.t -> date)
(** Return a function that reads an untagged
biniou value of type {!date}. *)
val read_date :
Bi_inbuf.t -> date
(** Input a tagged biniou value of type {!date}. *)
val date_of_string :
?pos:int -> string -> date
(** Deserialize a biniou value of type {!date}.
@param pos specifies the position where
reading starts. Default: 0. *)
(* Writers for type profile *)
val profile_tag : Bi_io.node_tag
(** Tag used by the writers for type {!profile}.
Readers may support more than just this tag. *)
val write_untagged_profile :
Bi_outbuf.t -> profile -> unit
(** Output an untagged biniou value of type {!profile}. *)
val write_profile :
Bi_outbuf.t -> profile -> unit
(** Output a biniou value of type {!profile}. *)
val string_of_profile :
?len:int -> profile -> string
(** Serialize a value of type {!profile} into
a biniou string. *)
(* Readers for type profile *)
val get_profile_reader :
Bi_io.node_tag -> (Bi_inbuf.t -> profile)
(** Return a function that reads an untagged
biniou value of type {!profile}. *)
val read_profile :
Bi_inbuf.t -> profile
(** Input a tagged biniou value of type {!profile}. *)
val profile_of_string :
?pos:int -> string -> profile
(** Deserialize a biniou value of type {!profile}.
@param pos specifies the position where
reading starts. Default: 0. *)
Module Example_t
(files example_t.mli
and example_t.ml
)
contains all OCaml type definitions that can be used independently from
Biniou or JSON.
For convenience, these definitions are also made available from the
Example_b
module whose interface is shown above. Any type name,
record field name or variant constructor can be referred to using either
module. For example, the OCaml expressions
((x : Example_t.date) : Example_b.date)
and
x.Example_t.year = x.Example_b.year
are both valid.
Validator example¶
$ atdgen -t example.atd
$ atdgen -v example.atd
Input file example.atd
:
type month = int <ocaml valid="fun x -> x >= 1 && x <= 12">
type day = int <ocaml valid="fun x -> x >= 1 && x <= 31">
type date = {
year : int;
month : month;
day : day;
}
<ocaml validator="Date_util.validate_date">
is used to produce files example_t.mli
, example_t.ml
,
example_v.mli
and example_v.ml
. This is example_v.ml
,
showing how the user-specified validators are used:
(* Auto-generated from "example.atd" *)
type gender = Example_t.gender
type date = Example_t.date = { year: int; month: int; day: int }
type profile = Example_t.profile = {
id: string;
email: string;
email_validated: bool;
name: string;
real_name: string option;
about_me: string list;
gender: gender option;
date_of_birth: date option
}
val validate_gender :
Atdgen_runtime.Util.Validation.path -> gender -> Atdgen_runtime.Util.Validation.error option
(** Validate a value of type {!gender}. *)
val create_date :
year: int ->
month: int ->
day: int ->
unit -> date
(** Create a record of type {!date}. *)
val validate_date :
Atdgen_runtime.Util.Validation.path -> date -> Atdgen_runtime.Util.Validation.error option
(** Validate a value of type {!date}. *)
val create_profile :
id: string ->
email: string ->
?email_validated: bool ->
name: string ->
?real_name: string ->
?about_me: string list ->
?gender: gender ->
?date_of_birth: date ->
unit -> profile
(** Create a record of type {!profile}. *)
val validate_profile :
Atdgen_runtime.Util.Validation.path -> profile -> Atdgen_runtime.Util.Validation.error option
(** Validate a value of type {!profile}. *)
Default type mapping¶
The following table summarizes the default mapping between ATD types and OCaml, biniou and JSON data types. For each language more representations are available and are detailed in the next section of this manual.
ATD | OCaml | JSON | Biniou |
---|---|---|---|
unit |
unit |
null | unit |
bool |
bool |
boolean | bool |
int |
int |
-?(0|[1-9][0-9]*) | svint |
float |
float |
number | float64 |
string |
string |
string | string |
'a option |
'a option |
"None" or
["Some", ...] |
numeric variants (tag 0) |
'a nullable |
'a option |
null or
representation of
'a |
numeric variants (tag 0) |
'a list |
'a list |
array | array |
'a shared |
no wrapping | not implemented | no longer supported |
'a wrap |
defined by
annotation,
converted from
'a |
representation of
'a |
representation of
'a |
variants | polymorphic variants | variants | regular variants |
record | record | object | record |
('a * 'b) |
('a * 'b) |
array | tuple |
('a) |
'a |
array | tuple |
Notes:
- Null JSON fields by default are treated as if the field was missing.
They can be made meaningful with the
keep_nulls
flag. - JSON nulls are used to represent the unit value and is useful for instanciating parametrized types with “nothing”.
- OCaml floats are written to JSON numbers with either a decimal point or an exponent such that they are distinguishable from ints, even though the JSON standard does not require a distinction between the two.
- The optional values of record fields denoted in ATD by a question mark are unwrapped or omitted in both biniou and JSON.
- JSON option values and JSON variants are represented in standard JSON
(
atdgen -j -j-std
) by a single string e.g."None"
or a pair in which the first element is the name (constructor) e.g.["Some", 1234]
. Yojson also provides a specific syntax for variants using edgy brackets:<"None">
,<"Some": 1234>
. - Biniou field names and variant names other than the option types use the hash of the ATD field or variant name and cannot currently be overridden by annotations.
- JSON tuples in standard JSON (
atdgen -j -j-std
) use the array notation e.g.["ABC", 123]
. Yojson also provides a specific syntax for tuples using parentheses, e.g.("ABC", 123)
. - Types defined as abstract are defined in another module.
ATD Annotations¶
Section json
¶
Field keep_nulls
¶
Position: after record
Values: none, true
or false
Semantics: this flag, if present or set to true, indicates that fields
whose JSON value is null
should not be treated as if they were
missing. In this case, null
is parsed as a normal value, possibly of
a nullable
type.
Example: patch semantics
(* Type of the objects stored in our database *)
type t = {
?x : int option;
?y : int option;
?z : int option;
}
(* Type of the requests to modify some of the fields of an object. *)
type t_patch = {
?x : int nullable option; (* OCaml type: int option option *)
?y : int nullable option;
?z : int nullable option;
} <ocaml field_prefix="patch_"> <json keep_nulls>
Let’s consider the following json patch that means “set x
to 1,
clear y
and keep z
as it is”:
{
"x": 1,
"y": null
}
It will be parsed by the generated function t_patch_of_string
into
the following OCaml value:
{
patch_x = Some (Some 1);
patch_y = Some None;
patch_z = None;
}
Then presumably some code would be written to apply the patch to an
object of type t
. Such code is not generated by atdgen at this time.
Available: from atd 1.12
Field name
¶
Position: after field name or variant name
Values: any string making a valid JSON string value
Semantics: specifies an alternate object field name or variant name to be used by the JSON representation.
Example:
type color = [
Black <json name="black">
| White <json name="white">
| Grey <json name="grey">
]
type profile = {
id <json name="ID"> : int;
username : string;
background_color : color;
}
A valid JSON object of the profile
type above is:
{
"ID": 12345678,
"username": "kimforever",
"background_color": "black"
}
Field repr
¶
Association lists¶
Position: after (string * _) list
type
Values: object
Semantics: uses JSON’s object notation to represent association lists.
Example:
type counts = (string * int) list <json repr="object">
A valid JSON object of the counts
type above is:
{
"bob": 3,
"john": 1408,
"mary": 450987,
"peter": 93087
}
Without the annotation <json repr="object">
, the data above would be
represented as:
[
[ "bob", 3 ],
[ "john", 1408 ],
[ "mary", 450987 ],
[ "peter", 93087 ]
]
Floats¶
Position: after float
type
Values: int
Semantics: specifies a float value that must be rounded to the nearest integer and represented in JSON without a decimal point nor an exponent.
Example:
type unixtime = float <json repr="int">
Ints¶
Position: after int
type
Values: string
Semantics: specifies a int value that must be represented in JSON as a string.
Example:
type int64 = int <ocaml repr="int64"> <json repr="string">
Field tag_field
¶
Superseded by <json adapter.ocaml="...">
. Available since atdgen
1.5.0 and yojson 1.2.0 until atdgen 1.13.
This feature makes it possible to read JSON objects representing variants that use one field for the tag and another field for the untagged value of the specific type associated with that tag.
Position: on a record field name, for a field holding a variant type.
Value: name of another JSON field which holds the string representing the constructor for the variant.
Semantics: The type definition
type t = {
value <json tag_field="kind">: [ A | B <json name="b"> of int ];
}
covers JSON objects that have an extra field kind
which holds either
"A"
or "b"
. Valid JSON values of type t
include
{ "kind": "A" }
and { "kind": "b", "value": 123 }
.
Field untyped
¶
Superseded by <json open_enum>
and <json adapter.ocaml="...">
.
Available since atdgen 1.10.0 and atd 1.2.0 until atdgen 1.13.
This flag enables parsing of arbitrary variants without prior knowledge
of their type. It is useful for constructing flexible parsers for
extensible serializations. json untyped
is compatible with regular
variants, json tag_field
variants, default values, and implicit
tag_field
constructors.
Position: on a variant constructor with argument type
string * json option
(at most one per variant type)
Value: none, true
or false
Semantics: The type definition
type v = [
| A
| B <json name="b"> of int
| Unknown <json untyped> of (string * json option)
]
will parse and print "A"
, ["b", 0]
, "foo"
, and
["bar", [null]]
in a regular variant context. In the tag_field
type t
context in the previous section, v
will parse and print
{ "kind": "foo" }
and { "kind": "bar", "value": [null] }
as well
as the examples previously given.
Field open_enum
¶
Where an enum (finite set of strings) is expected, this flag allows unexpected strings to be kept under a catch-all constructor rather than producing an error.
Position: on a variant type comprising exactly one constructor with an
argument. The type of that argument must be string
. All other
constructors must have no arguments.
Value: none
For example:
type language = [
| English
| Chinese
| Other of string
] <json open_enum>
maps the json string "Chinese"
to the OCaml value `Chinese
and
maps "French"
to `Other "French"
.
Available since atdgen 2.0.
Field adapter.ocaml
¶
Json adapters are a mechanism for rearranging json data on-the-fly, so
as to make them compatible with ATD. The programmer must provide an
OCaml module that provides converters between the original json
representation and the ATD-compatible representation. The signature of
the user-provided module must be equal to
Atdgen_runtime.Json_adapter.S
, which is:
sig
(** Convert from original json to ATD-compatible json *)
val normalize : Yojson.Safe.t -> Yojson.Safe.t
(** Convert from ATD-compatible json to original json *)
val restore : Yojson.Safe.t -> Yojson.Safe.t
end
The type Yojson.Safe.t
is the type of parsed JSON as provided by
the yojson library.
Position: on a variant type or on a record type.
Value: an OCaml module identifier. Note that
Atdgen_runtime.Json_adapter
provides a few modules and functors that
are ready to use. Users are however encouraged to write their own to
suit their needs.
Sample ATD definitions:
type document = [
| Image of image
| Text of text
] <json adapter.ocaml="Atdgen_runtime.Json_adapter.Type_field">
type image = {
url: string;
}
type text = {
title: string;
body: string;
}
ATD-compliant json values:
["Image", {"url": "https://example.com/ocean123.jpg"}]
["Text", {"title": "Cheeses Around the World", "body": "..."}]
Corresponding json values given by some API:
{"type": "Image", "url": "https://example.com/ocean123.jpg"}
{"type": "Text", "title": "Cheeses Around the World", "body": "..."}
The json adapter Type_field
that ships with the atdgen runtime takes
care of converting between these two forms. For information on how to
write your own adapter, please consult the documentation for the yojson
library.
Fields adapter.to_ocaml
and adapter.from_ocaml
¶
This is an alternative form of specifying adapter.ocaml
. It permits to
specify arbitrary code and doesn’t require the “adapter” module to be defined
in advance.
For example, the above usage of adapter.ocaml
can be rewritten as
following:
type document = [
| Image of image
| Text of text
]
<json
adapter.to_ocaml="Atdgen_runtime.Json_adapter.normalize_type_field \"type\""
adapter.from_ocaml="Atdgen_runtime.Json_adapter.restore_type_field \"type\""
>
type image = {
url: string;
}
type text = {
title: string;
body: string;
}
Section biniou
¶
Field repr
¶
Integers¶
Position: after int
type
Values: svint
(default), uvint
, int8
, int16
, int32
,
int64
Semantics: specifies an alternate type for representing integers. The
default type is svint
. The other integers types provided by biniou
are supported by Atdgen-biniou. They have to map to the corresponding
OCaml types in accordance with the following table:
Biniou type | Supported OCaml type | OCaml value range |
---|---|---|
svint |
int |
min_int … max_int |
uvint |
int |
0 … max_int , min_int … -1 |
int8 |
char |
'\000 … '\255 |
int16 |
int |
0 … 65535 |
int32 |
int32 |
Int32.min_int …
Int32.max_int |
int64 |
int64 |
Int64.min_int …
Int64.max_int |
In addition to the mapping above, if the OCaml type is int
, any
biniou integer type can be read into OCaml data regardless of the
declared biniou type.
Example:
type t = {
id : int
<ocaml repr="int64">
<biniou repr="int64">;
data : string list;
}
Floating-point numbers¶
Position: after float
type
Values: float64
(default), float32
Semantics: float32
allows for a shorter serialized representation of
floats, using 4 bytes instead of 8, with reduced precision. OCaml floats
always use 8 bytes, though.
Example:
type t = {
lat : float <biniou repr="float32">;
lon : float <biniou repr="float32">;
}
Arrays and tables¶
Position: applies to lists of records
Values: array
(default), table
Semantics: table
uses biniou’s table format instead of a regular
array for serializing OCaml data into biniou. Both formats are supported
for reading into OCaml data regardless of the annotation. The table
format allows
Example:
type item = {
id : int;
data : string list;
}
type items = item list <biniou repr="table">
Section ocaml
¶
Field attr
¶
Position: on a type definition, i.e. on the left-handside just before
the equal sign =
Semantics: specifies custom ppx attributes for the type
definition. Overrides any default attributes set globally via
the command line option -type-attr
.
Values: the contents of a ppx annotation without the enclosing
[@@
and ]
Example:
type foo <ocaml attr="deriving show,eq"> = int list
translates to
type foo = int list [@@deriving show,eq]
Field predef
¶
Position: left-hand side of a type definition, after the type name
Values: none, true
or false
Semantics: this flag indicates that the corresponding OCaml type definition must be omitted.
Example:
(* Some third-party OCaml code *)
type message = {
from : string;
subject : string;
body : string;
}
(*
Our own ATD file used for making message_of_string and
string_of_message functions.
*)
type message <ocaml predef> = {
from : string;
subject : string;
body : string;
}
Field mutable
¶
Position: after a record field name
Values: none, true
or false
Semantics: this flag indicates that the corresponding OCaml record field is mutable.
Example:
type counter = {
total <ocaml mutable> : int;
errors <ocaml mutable> : int;
}
translates to the following OCaml definition:
type counter = {
mutable total : int;
mutable errors : int;
}
Field default
¶
Position: after a record field name marked with a \~{
} symbol or at
the beginning of a tuple field.
Values: any valid OCaml expression
Semantics: specifies an explicit default value for a field of an OCaml record or tuple, allowing that field to be omitted. Default strings must be escaped.
Example:
type color = [ Black | White | Rgb of (int * int * int) ]
type ford_t = {
year : int;
~color <ocaml default="`Black"> : color;
~name <ocaml default="\"Ford Model T\""> : string;
}
type point = (int * int * <ocaml default="0"> : int)
Field from
¶
Position: left-hand side of a type definition, after the type name
Values: OCaml module name without the _t
, _b
, _j
or _v
suffix. This can be also seen as the name of the original ATD file,
without the .atd
extension and capitalized like an OCaml module
name.
Semantics: specifies the base name of the OCaml modules where the type and values coming with that type are defined.
It is useful for ATD types defined as abstract
and for types
annotated as predefined using the annotation <ocaml predef>
. In both
cases, the missing definitions must be provided by modules composed of
the base name and the standard suffix assumed by Atdgen which is _t
,
_b
, _j
or _v
.
Example: First input file part1.atd
:
type point = { x : int; y : int }
Second input file part2.atd
depending on the first one:
type point <ocaml from="Part1"> = abstract
type points = point list
To use a different type name than defined in the Part1
module, add a
t
field declaration to the annotation which refers to the original
type name:
type point_xy <ocaml from="Part1" t="point"> = abstract
type points = point_xy list
Field module
¶
Using a custom wrapper¶
Using the built-in wrap
constructor, it is possible to add a layer
of abstraction on top of the concrete structure used for serialization.
Position: after a wrap
type constructor
Values: OCaml module name
A common use case is to parse strings used as unique identifiers and
wrap the result into an abstract type. Our OCaml module Uid
needs to
provide a type t
, and two functions wrap
and unwrap
as
follows:
type t
val wrap : string -> t
val unwrap : t -> string
Given that Uid
OCaml module, we can write the following ATD
definition:
type uid = string wrap <ocaml module="Uid">
Other languages than OCaml using the same ATD type definitions may or
may not add their own abstract layer. Without an annotation, the
wrap
construct has no effect on the value being wrapped, i.e.
wrap
and unwrap
default to the identity function.
It is also possible to define t
, wrap
, and unwrap
inline:
type uid = string wrap <ocaml t="Uid.t"
wrap="Uid.wrap"
unwrap="Uid.unwrap">
This can be useful for very simple validation:
type uid = string wrap
<ocaml wrap="fun s ->
if String.length s <> 16 then
failwith \"Invalid user ID\";
s"
>
Importing an external type definition¶
In most cases since Atdgen 1.2.0 module
annotations are deprecated
in favor of from
annotations previously described.
Position: left-hand side of a type definition, after the type name
Values: OCaml module name
Semantics: specifies the OCaml module where the type and values coming
with that type are defined. It is useful for ATD types defined as
abstract
and for types annotated as predefined using the annotation
<ocaml predef>
. In both cases, the missing definitions can be
provided either by globally opening an OCaml module with an OCaml
directive or by specifying locally the name of the module to use.
The latter approach is recommended because it allows to create type and value aliases in the OCaml module being generated. It results in a complete module signature regardless of the external nature of some items.
Example: Input file example.atd
:
type document <ocaml module="Doc"> = abstract
type color <ocaml predef module="Color"> =
[ Black | White ] <ocaml repr="classic">
type point <ocaml predef module="Point"> = {
x : float;
y : float;
}
gives the following OCaml type definitions (file example.mli
):
type document = Doc.document
type color = Color.color = Black | White
type point = Point.point = { x: float; y: float }
Now for instance Example.Black
and Color.Black
can be used
interchangeably in other modules.
Field t
¶
Using a custom wrapper¶
Specifies the OCaml type of an abstract wrap
construct, possibly
overriding the default M.t
if M is the module where the
wrap
and unwrap
functions are found.
Position: after a wrap
type constructor
Values: OCaml type name
Example:
type uid = string wrap <ocaml module="Uid" t="Uid.uid">
is equivalent to:
type uid = string wrap <ocaml t="Uid.uid" wrap="Uid.wrap" unwrap="Uid.unwrap">
Importing an external type definition¶
Position: left-hand side of a type definition, after the type name. Must
be used in conjunction with a module
field.
Values: OCaml type name as found in an external module.
Semantics: This option allows to specify the name of an OCaml type defined in an external module.
It is useful when the type needs to be renamed because its original name
is already in use or not enough informative. Typically we may want to
give the name foo
to a type originally defined in OCaml as
Foo.t
.
Example:
type foo <ocaml_biniou module="Foo" t="t"> = abstract
type bar <ocaml_biniou module="Bar" t="t"> = abstract
type t <ocaml_biniou module="Baz"> = abstract
allows local type names to be unique and gives the following OCaml type definitions:
type foo = Foo.t
type bar = Bar.t
type t = Baz.t
Fields wrap
and unwrap
¶
See “Using a custom wrapper” under section ocaml
, fields
module
and t
.
Field field_prefix
¶
Position: record type expression
Values: any string making a valid prefix for OCaml record field names
Semantics: specifies a prefix to be prepended to each field of the OCaml definition of the record. Overridden by alternate field names defined on a per-field basis.
Example:
type point2 = {
x : int;
y : int;
} <ocaml field_prefix="p2_">
gives the following OCaml type definition:
type point2 = {
p2_x : int;
p2_y : int;
}
Field name
¶
Position: after record field name or variant name
Values: any string making a valid OCaml record field name or variant name
Semantics: specifies an alternate record field name or variant names to be used in OCaml.
Example:
type color = [
Black <ocaml name="Grey0">
| White <ocaml name="Grey100">
| Grey <ocaml name="Grey50">
]
type profile = {
id <ocaml name="profile_id"> : int;
username : string;
}
gives the following OCaml type definitions:
type color = [
`Grey0
| `Grey100
| `Grey50
]
type profile = {
profile_id : int;
username : string;
}
Field repr
¶
Integers¶
Position: after int
type
Values: char
, int32
, int64
, float
Semantics: specifies an alternate type for representing integers. The
default type is int
, but char
, int32
, int64
or float
can be used instead.
The three types char
, int32
and int64
are supported by both
Atdgen-biniou and Atdgen-json but Atdgen-biniou currently requires that
they map to the corresponding fixed-width types provided by the biniou
format.
The type float
is only supported in conjunction with JSON and is
useful when an OCaml float is used to represent an integral value, such
as a time in seconds returned by Unix.time()
. When converted into
JSON, floats are rounded to the nearest integer.
Example:
type t = {
id : int
<ocaml repr="int64">
<biniou repr="int64">;
data : string list;
}
Lists and arrays¶
Position: after a list
type
Values: array
Semantics: maps to OCaml’s array
type instead of list
.
Example:
type t = {
id : int;
data : string list
<ocaml repr="array">;
}
Sum types¶
Position: after a sum type (denoted by square brackets)
Values: classic
Semantics: maps to OCaml’s classic variants instead of polymorphic variants.
Example:
type fruit = [ Apple | Orange ] <ocaml repr="classic">
translates to the following OCaml type definition:
type fruit = Apple | Orange
Field valid
¶
Since atdgen 1.6.0.
Position: after any type expression except type variables
Values: OCaml function that takes one argument of the given type and returns a bool
Semantics: atdgen -v
produces for each type named t a function
validate_
t:
val validate_t : Atdgen_runtime.Util.Validation.path -> t -> Atdgen_runtime.Util.Validation.error option
Such a function returns None
if and only if the value and all of its
subnodes pass all the validators specified by annotations of the form
<ocaml validator="...">
or <ocaml valid="...">
(at most one per
node).
Example:
type positive = int <ocaml validator="fun x -> x > 0">
type point = {
x : positive;
y : positive;
z : int;
}
<ocaml valid="Point.validate">
(* Some validating function from a user-defined module Point *)
The generated validate_point
function calls the validator for the
containing object first (Point.validate
) and continues on its fields
x
then y
until an error is returned.
match validate_point [] { x = 1; y = 0; z = 1 } with
| None -> ()
| Some e ->
Printf.eprintf "Error: %s\n%!"
(Atdgen_runtime.Util.Validation.string_of_error e)
The above code prints the following error message:
Error: Validation error; path = <root>.y
In order to customize the error message and print the faulty value, use
validator
instead of valid
, as described next.
Field validator
¶
This is a variant of the valid
annotation that allows full control
over the error message that gets generated in case of an error.
Position: after any type expression except type variables
Values: OCaml function that takes the path in current JSON structure and the object to validate, and returns an optional error.
Semantics: atdgen -v
produces for each type named t a function
validate_
t:
val validate_t : Atdgen_runtime.Util.Validation.path -> t -> Atdgen_runtime.Util.Validation.error option
Such a function returns None
if and only if the value and all of its
subnodes pass all the validators specified by annotations of the form
<ocaml validator="...">
or <ocaml valid="...">
(at most one per
node).
Example:
type positive = int <ocaml validator="
fun path x ->
if x > 0 then None
else
Some (
Atdgen_runtime.Util.Validation.error
~msg: (\"Not a positive integer: \" ^ string_of_int x)
path
)
">
type point = {
x : positive;
y : positive;
z : int;
}
<ocaml validator="Point.validate">
(* Some validating function from a user-defined module Point *)
The following user code
match Toto_v.validate_point [] { x = 1; y = 0; z = 1 } with
| None -> ()
| Some e ->
Printf.eprintf "Error: %s\n%!"
(Atdgen_runtime.Util.Validation.string_of_error e)
results in printing:
Error: Validation error: Not a positive integer: 0; path = <root>.y
Section ocaml_biniou
¶
Section ocaml_biniou
takes precedence over section ocaml
in
Biniou mode (-b
) for the following fields:
predef
(see sectionocaml
, fieldpredef
)module
(see sectionocaml
, fieldmodule
)t
(see sectionocaml.t
)
Section ocaml_json
(obsolete)¶
Section ocaml_json
takes precedence over section ocaml
in JSON
mode (-json
or -j
) for the following fields:
predef
(see sectionocaml
, fieldpredef
)module
(see sectionocaml
, fieldmodule
)t
(see sectionocaml
, fieldt
)
Please note that atdgen -json
is now deprecated in favor of
atdgen -j
(json) and atdgen -t
(types). The latter is in charge
of producing type definitions independently from JSON and will ignore
<ocaml_json ...>
annotations, making them almost useless. The
equivalent <ocaml ...>
annotations are almost always preferable.
Example:
This example shows how to parse a field into a generic tree of type
Yojson.Safe.t
rather than a value of a specialized OCaml type.
type dyn <ocaml_json module="Yojson.Safe" t="json"> = abstract
type t = { foo: int; bar: dyn }
translates to the following OCaml type definitions:
type dyn = Yojson.Safe.t
type t = { foo : int; bar : dyn }
Sample OCaml value of type t
:
{
foo = 12345;
bar =
`List [
`Int 12;
`String "abc";
`Assoc [
"x", `Float 3.14;
"y", `Float 0.0;
"color", `List [ `Float 0.3; `Float 0.0; `Float 1.0 ]
]
]
}
Corresponding JSON data as obtained with string_of_t
:
{"foo":12345,"bar":[12,"abc",{"x":3.14,"y":0.0,"color":[0.3,0.0,1.0]}]}
Section doc
¶
Unlike comments, doc
annotations are meant to be propagated into the
generated source code. This is useful for making generated interface
files readable without having to consult the original ATD file.
Generated source code comments can comply to a standard format and take advantage of documentation generators such as javadoc or ocamldoc.
Field text
¶
Position:
- after the type name on the left-hand side of a type definition
- after the type expression on the right hand of a type definition (but not after any type expression)
- after record field names
- after variant names
Values: UTF-8-encoded text using a minimalistic markup language
Semantics: The markup language is defined as follows:
- Blank lines separate paragraphs.
{{ }}
can be used to enclose inline verbatim text.{{{ }}}
can be used to enclose verbatim text where whitespace is preserved.- The backslash character is used to escape special character
sequences. In regular paragraph mode the special sequences are
\
,{{
and{{{
. In inline verbatim text, special sequences are\
and}}
. In verbatim text, special sequences are\
and}}}
.
Example: The following is an example demonstrating the use of doc
annotations generated using:
$ atdgen -t ocamldoc_example.atd
Input file ocamldoc_example.atd
:
<doc text="This is the title">
type point = {
x <doc text="The first coordinate">: float;
y <doc text="The second coordinate">: float;
}
<doc text="
The type of a point. A value {{p}} can be created as follows:
{{{
let p = { x = 1.2; y = 5.0 }
}}}
">
type color = [
| Black <doc text="Same as {{RGB (0,0,0)}}">
| White <doc text="Same as {{RGB (255, 255, 255)}}">
| RGB
<doc text="Red, green, blue components">
of (int * int * int)
]
translates using atdgen -t ocamldoc_example.atd
into the following
OCaml interface file ocamldoc_example_t.mli
with ocamldoc-compliant
comments:
(* Auto-generated from "ocamldoc_example.atd" *)
(** This is the title *)
(**
The type of a point. A value [p] can be created as follows:
{v
let p = \{ x = 1.2; y = 5.0 \}
v}
*)
type point = {
x: float (** The first coordinate *);
y: float (** The second coordinate *)
}
type color = [
`Black (** Same as [RGB (0,0,0)] *)
| `White (** Same as [RGB (255, 255, 255)] *)
| `RGB of (int * int * int) (** Red, green, blue components *)
]
Atdgen runtime library¶
A library named atdgen-runtime is installed by the standard installation process. Only a fraction of it is officially supported and documented.
Modules intended for all users are:
Util
Json_adapter
The other modules exported by the library are used directly by generated code. Tool developers may use them but we don’t guarantee strong compatibility across releases.