Getting Started
*Please be advised that CoreLogic is switching the Trestle™ API URL to a new URL
Old: https://api-prod.corelogic.com
New: https://api-trestle.corelogic.com
The Old URL will continue to work until February 6th 2024.
https://api-trestle.corelogic.com
Video Tutorial
For the 2020 RESO Remote digital tech summit, Al McElmon put together a video tutorial on building a WebAPI client to work with Trestle. You can watch the video below along with reading this Getting Started guide.
Introduction
Note: The provided code samples exclude most error-handling for brevity's sake. We do recommend adding in error-handling if you use these samples in your production code.
Trestle’s Web API is based on and compliant with the RESO Web API standard and the RESO Data Dictionary standard. The RESO Web API standard is based on OData and provides a unified way to access data across multiple platforms. Features of Trestle’s Web API that extend RESO’s standard are called out and highlighted.
If you do not already have a Web API feed set up, Trestle offers a sample feed for you to use to get acclimated to the API and its responses.
Trestle's WebAPI Sample Feed is provided by Austin Board of Realtors
You may also be interested in the RETS documentation if you are not ready for Web API.
Below, we will demonstrate how to set up a basic Trestle client that replicates property data and media. You will learn how to authenticate using OAuth2, how to fetch the metadata to learn what data you can get, how to fetch the property data and the related media files, as well as a few other tips and tricks for working with Trestle via WebAPI.
To follow along without writing code, you can use Postman to run the queries
Authentication
POST /trestle/oidc/connect/token HTTP/1.1
Host: api-trestle.corelogic.com
Content-Type: application/x-www-form-urlencoded
client_id=<CLIENT_ID>&client_secret=<CLIENT_SECRET>&grant_type=client_credentials&scope=api
private static async Task Authenticate(string clientId, string clientSecret)
{
var stringContent = new FormUrlEncodedContent(new[]
{
new KeyValuePair<string, string>("client_id", clientId),
new KeyValuePair<string, string>("client_secret", clientSecret),
new KeyValuePair<string, string>("grant_type", "client_credentials"),
new KeyValuePair<string, string>("scope", "api"),
});
var tokResp = await Client.PostAsync("https://api-trestle.corelogic.com/trestle/oidc/connect/token",
stringContent);
var tokBody = await tokResp.Content.ReadAsStringAsync();
var tokJson = JsonDocument.Parse(tokBody);
var token = tokJson.RootElement.GetProperty("access_token").GetString();
Client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token);
}
import com.google.gson.Gson;
import org.apache.http.client.fluent.Form;
import org.apache.http.client.fluent.Request;
String content = Request.Post("https://api-trestle.corelogic.com/trestle/oidc/connect/token")
.bodyForm(Form.form()
.add("client_id", client_id)
.add("client_secret", client_secret)
.add("scope", "api")
.add("grant_type", "client_credentials")
.build()
).execute().returnContent().asString();
OAuth2Token token = new Gson().fromJson(content, OAuth2Token.class);
import (
"golang.org/x/oauth2"
"golang.org/x/oauth2/clientcredentials"
)
conf := &clientcredentials.Config{
ClientID: ClientId,
ClientSecret: ClientSecret,
TokenURL: "https://api-trestle.corelogic.com/trestle/oidc/connect/token",
Scopes: []string{"api"},
EndpointParams: nil,
AuthStyle: oauth2.AuthStyleInParams,
}
client := conf.Client(context.Background())
const oauth2 = require('simple-oauth2');
async function authenticate(client_id, client_secret) {
const trestleConfig = {
client: {
id: client_id,
secret: client_secret
},
auth: {
tokenHost: "https://api-trestle.corelogic.com",
tokenPath: "/trestle/oidc/connect/token"
}
};
try {
const o = new oauth2.ClientCredentials(trestleConfig);
return await o.getToken({scope: 'api'});
/* The above is for the latest version of simple-oauth2
If you are still using a 3.x version, the code will need
to look like this:
const o = oauth2.create(trestleConfig);
const result = await o.clientCredentials.getToken({scope: 'api'});
return o.accessToken.create(result);
*/
} catch (error) {
console.log(error);
}
}
from oauthlib.oauth2 import BackendApplicationClient
from requests_oauthlib import OAuth2Session
c = BackendApplicationClient(client_id=client_id)
session = OAuth2Session(client=c)
session.fetch_token(token_url='https://api-trestle.corelogic.com/trestle/oidc/connect/token',
client_id=client_id, client_secret=client_secret,
scope='api')
extern crate reqwest;
extern crate serde_json;
use std::collections::HashMap;
pub async fn get_token(client_id: &str, client_secret: &str, token_uri: &str, scope: &str)
-> Result<String, reqwest::Error> {
let mut map = HashMap::new();
map.insert("client_id", client_id);
map.insert("client_secret", client_secret);
map.insert("grant_type", "client_credentials");
map.insert("scope", scope);
let client = reqwest::Client::new();
let body = client.post(token_uri)
.form(&map)
.send().await?
.text().await?;
let v : serde_json::Value = serde_json::from_str(&body).unwrap();
Ok(v["access_token"].as_str().unwrap().to_string())
}
You will get a JSON response like this:
{
"access_token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1Ni...3wQJVBA",
"expires_in": 28800,
"token_type": "Bearer"
}
In order to do anything in Trestle, you must first be authenticated. Trestle’s Web API uses an OAuth2 Client Credentials flow for authentication. Once you have obtained a token, it is good for up to 8 hours. We recommend caching the token and its expiration and refreshing it only when necessary.
/trestle/oidc/connect/token
To authenticate, you will need to POST a x-www-form-urlencoded message to https://api-trestle.corelogic.com/trestle/oidc/connect/token
. This message has four values:
Value | Description |
---|---|
client_id | Your username or client id as gotten from Trestle |
client_secret | Your password as gotten from Trestle |
scope | The scope for your authentication. This should always be api for WebAPI. |
grant_type | The type of oauth2 grant. This should always be client_credentials . |
Your message will look like client_id=trestle_helpsite&client_secret=d34db33f&scope=api&grant_type=client_credentials
.
A successful authentication request will return a JSON object with three values:
Property | Description |
---|---|
access_token | This is the authorization token that you will pass in the Authorization header in subsequent requests. |
expires_in | This value indicates how long the token is valid for, in seconds. As of now, it will always be 28800 . |
token_type | This value indicates what kind of token is being returned. As of now, it will always be Bearer . |
The access token you get will be usable for 8 hours. You can request a new one more frequently if needed.
Reading the Metadata
GET /odata/$metadata HTTP/1.1
Host: https://api-trestle.corelogic.com/trestle
Accept: application/xml
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1Ni...3wQJVBA
public static async Task Main(string[] args)
{
Client.BaseAddress = new Uri("https://api-trestle.corelogic.com/trestle/odata/");
await Authenticate(clientId, clientSecret);
var xmlDocument = await GetMetadata();
/* Example of using xpath query to get information from the metadata
var xmlNamespaceManager = new XmlNamespaceManager(xmlDocument.NameTable);
xmlNamespaceManager.AddNamespace("od", "http://docs.oasis-open.org/odata/ns/edm");
const string xpath = "//od:EntityContainer/od:EntitySet";
var root = xmlDocument.DocumentElement;
var nodeList = root.SelectNodes(xpath, xmlNamespaceManager);
foreach (XmlNode entitySet in nodeList)
{
Console.WriteLine(entitySet.Attributes["Name"].InnerText);
}
*/
}
private static async Task<XmlDocument> GetMetadata()
{
var response = await Client.GetAsync("$metadata");
var body = await response.Content.ReadAsStringAsync();
var xmlDocument = new XmlDocument();
xmlDocument.LoadXml(body);
return xmlDocument;
}
import org.apache.olingo.client.api.ODataClient;
import org.apache.olingo.client.api.communication.request.retrieve.EdmMetadataRequest;
import org.apache.olingo.commons.api.edm.Edm;
ODataClient oDataClient = ODataClientFactory.getClient();
EdmMetadataRequest edmMetadataRequest = oDataClient.getRetrieveRequestFactory().getMetadataRequest(service_uri);
Edm metadata = edmMetadataRequest.execute().getBody();
import (
"encoding/xml"
"io/ioutil"
)
type DataServices struct {
Schemas []Schema `xml:"Schema"`
}
type Metadata struct {
Version string `xml:"Version,attr"`
DataServices DataServices
}
func (o *Odata) FetchMetadata() *Metadata {
u := *o.BaseURL
u.Path += "$metadata"
resp, _ := o.Client.Get(u.String())
defer resp.Body.Close()
var m Metadata
data, _ := ioutil.ReadAll(resp.Body)
_ = xml.Unmarshal(data, &m)
return &m
}
const odata = require('odata');
const xmljs = require('xml-js');
const accessToken = await authenticate(client_id, client_secret);
const trestle = odata.o('https://api-trestle.corelogic.com/trestle/odata/', {
headers: {
'Authorization': 'Bearer ' + accessToken.token.access_token
},
fragment: "" // fragment defaults to "value" which just returns the data
}); // we often need odata.nextLink, so we specify "" to get
// all of the response
let response = await trestle
.get('$metadata')
.fetch()
.catch(e => console.log(e));
let text = await response.text();
let metadata = xmljs.xml2json(text, {compact: false, spaces: 4});
import untangle
r = session.get(f"{base_uri}/$metadata")
metadata = untangle.parse(r.text)
extern crate reqwest;
extern crate tokio;
extern crate roxmltree;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let token = get_raw_token(CLIENT_ID, CLIENT_SECRET, TOKEN_URI, "api").await?;
let mut auth_header : String = "Bearer ".to_owned();
auth_header.push_str(token.as_str());
let client = reqwest::Client::new();
let resp = client.get("https://api-trestle.corelogic.com/trestle/odata/$metadata")
.header("Authorization", auth_header)
.send().await?
.text().await?;
let doc = roxmltree::Document::parse(resp.as_str()).unwrap();
for node in doc.descendants().filter(|n| n.has_tag_name("Schema")) {
println!("{}", node.attribute("Namespace").unwrap());
}
Ok(())
}
Note that the Metadata endpoint returns XML, not JSON
<edmx:Edmx Version="4.0" xmlns:edmx="http://docs.oasis-open.org/odata/ns/edmx">
<edmx:DataServices>
<Schema Namespace="CoreLogic.DataStandard.RESO.DD" xmlns="http://docs.oasis-open.org/odata/ns/edm">
<EntityType Name="Property">
<Key>
<PropertyRef Name="ListingKey" />
</Key>
<Property Name="AboveGradeFinishedArea" Type="Edm.Decimal" Precision="14" Scale="2" />
</EntityType>
</Schema>
<Schema Namespace="CoreLogic.DataStandard.RESO.DD.Enums" xmlns="http://docs.oasis-open.org/odata/ns/edm">
<EnumType Name="AreaSource" UnderlyingType="Edm.Int64">
<Member Name="ListingAgent" Value="4">
<Annotation Term="RESO.OData.Metadata.TrestleName" String="Listing Agent" />
</Member>
</EnumType>
</Schema>
</edmx:DataServices>
</edmx:Edmx>
Once you've authenticated, the next step for building a client is finding out what data you can get out of Trestle. To do this, you query the metadata endpoint.
/trestle/odata/$metadata
The metadata for WebAPI conforms to OData's metadata definition, which is based on Microsoft's Entity Data Model. It is broken up into five schemas, each with its own namespace:
Schema | Description |
---|---|
CoreLogic.DataStandard.RESO.DD | Describes the RESO resources and the fields within them that are queryable through the WebAPI interface. |
CoreLogic.DataStandard.RESO.WebAPI | Describes the shape of the data available from the DataSystem endpoint. |
CoreLogic.DataStandard.RESO.DD.Enums | Describes the enumerations and their values. |
Default | Describes the available endpoints, including the $expand -able endpoints. |
For more information on the data contained in the metadata, please see the Metadata browser.
Fetching Property Data
GET /odata/Property HTTP/1.1
Host: https://api-trestle.corelogic.com/trestle
Accept: application/json
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1Ni...3wQJVBA
public static async Task Main(string[] args)
{
Client.BaseAddress = new Uri("https://api-trestle.corelogic.com/trestle/odata/");
await Authenticate(clientId, clientSecret);
var jsonDocument = await GetData("Property", null);
/* Example of how to iterate through the values returned
var values = jsonDocument.RootElement.GetProperty("value");
for (var i = 0; i < values.GetArrayLength(); i++)
{
Console.WriteLine(values[i].GetProperty("ListingKey").GetString());
}
*/
}
private static async Task<JsonDocument> GetData(string resource, string query)
{
var uri = resource;
if (query != null)
{
uri += "?" + query;
}
var response = await Client.GetAsync(uri);
var body = await response.Content.ReadAsStringAsync();
return JsonDocument.Parse(body);
}
import org.apache.olingo.client.api.ODataClient;
import org.apache.olingo.client.api.communication.request.retrieve.ODataEntitySetIteratorRequest;
import org.apache.olingo.client.api.domain.ClientEntity;
import org.apache.olingo.client.api.domain.ClientEntitySet;
import java.net.URI;
ODataClient oDataClient = ODataClientFactory.getClient();
URI uri = oDataClient.newURIBuilder(service_uri)
.appendEntitySetSegment("Property")
.build();
ODataEntitySetIteratorRequest<ClientEntitySet, ClientEntity> request =
oDataClient.getRetrieveRequestFactory().getEntitySetIteratorRequest(uri);
import (
"encoding/json"
"golang.org/x/oauth2"
"golang.org/x/oauth2/clientcredentials"
"io/ioutil"
)
type OdataResponse struct {
OdataContext string `json:"@odata.context"`
OdataNextLink string `json:"@odata.nextLink"`
OdataCount int `json:"@odata.count"`
Values []map[string]interface{} `json:"value"`
}
client := conf.Client(context.Background())
resp, _ := client.Get(base_uri + "/Property")
var r OdataResponse
data, _ := ioutil.ReadAll(resp.Body)
_ = json.Unmarshal(data, &r)
let response = await trestle
.get("Property")
.query()
.catch(e => console.log(e));
console.log(response);
r = session.get(f"{base_uri}/Property")
properties = r.json()['value']
extern crate reqwest;
extern crate tokio;
extern crate serde_json;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let token = get_raw_token(CLIENT_ID, CLIENT_SECRET, TOKEN_URI, "api").await?;
let mut auth_header : String = "Bearer ".to_owned();
auth_header.push_str(token.as_str());
let client = reqwest::Client::new();
let resp = client.get("https://api-trestle.corelogic.com/trestle/odata/Property")
.header("Authorization", auth_header)
.send().await?
.text().await?;
let v : serde_json::Value = serde_json::from_str(&resp).unwrap();
println!("{}", v["value"][0]["ListingKey"].as_str().unwrap());
Ok(())
}
The property data is returned in the
value
array in the OData response
{
"@odata.context": "https://api-trestle.corelogic.com/trestle/odata/$metadata#Property/CoreLogic.DataStandard.RESO.DD.Property",
"@odata.nextLink": "https://api-trestle.corelogic.com/trestle/odata/Property?$skip=10",
"value": [ ... ]
}
Now that we know what resources, fields, and enumerations are available via Trestle, it's time to start fetching property data.
/trestle/odata/Property
The first step is a basic query with no parameters. By default, Trestle returns 10 records per query--we'll address how to get more records later. If this is your first foray into WebAPI, it will be enough for you to learn about the structure of the data that's returned.
The basic query will return a JSON object with three components:
Value | Description |
---|---|
@odata.context |
References the portion of the metadata that describes the data being returned |
@odata.nextLink |
URI for getting the next set of records. Will only be present when there are more records to retrieve. |
value |
An array of objects. The data you came for. |
While you look through the data you have pulled back, there is one field that you should pay special attention to: ListingKey. This field is the unique identifier for the Property record. While ListingId might look more familiar, we do not guarantee its uniqueness.
ListingKey is the unique identifier for Property records
If you'd like to start playing with getting more data, you can skip ahead to the sections on $top and $skip.
If you'd like to start playing with getting specific records, you can look at the section on $filter.
If you'd like to start refining the fields that are returned to you, you can jump to the section on $select.
Fetching Media
Now that we have property data, we want to get some photos to go along with that data. There are two main ways to get photos from Trestle and we will look at both.
The first method we will look at is fetching the photos individually by unique URLs. After that, we will look at fetching all of the photos for one listing ine one query.
Fetching Media URLs
To get the images for a specific property, we will need the property's ListingKey (remember that ListingKey is unique for properties). The corresponding key in the Media resource is called ResourceRecordKey.
/trestle/odata/Media
We will construct a query like /trestle/odata/Media?$filter=ResourceRecordKey eq '123456'&$orderby=Order
. There's a good bit to unpack there, and we will get into it more in the $filter and $orderby sections. For now, just know that this is filtering the Media resource to only return records where ResourceRecordKey
equals "123456" and returns them in order.
There are many useful fields that are returned, but the one we care about right now is MediaURL
. The value of this field is a full URL that you can use to download an image from Trestle.
Fetching bulk Media
--UniqueBoundary
Content-ID: 123456
Object-ID: 1
OrderHint: 1
Content-Type: image/jpeg
imagedata
--UniqueBoundary
...
--UniqueBoundary--
/trestle/odata/Property('ListingKey
')/Media/All
Fetching images one at a time can work, but we also want to be able to fetch all of the images for a listing in one query. To do this, we will query the Media/All endpoint like: /trestle/odata/Property('123456')/Media/All
. If you are familiar with RETS, then you will recognize the results of this query as being the same as a GetObject wildcard query. It is a mime multipart response and will require parsing the output to get the images.
To help you, we provide a collection of mime multipart parsers in various languages
The response format starts with a unique boundary marker, e.g. --TRM27d89073c5c743c0a76a84cabd085e30
. This boundary marker indicates the beginning of a new image within the response. There is one final marker --TRM27d89073c5c743c0a76a84cabd085e30--
indicating the end of the images.
After the boundary marker are headers describing the image. In Trestle, we provide the ListingKey as Content-ID
, the order of the image as both Object-ID
and OrderHint
and the type of image as Content-Type
.
There is then a blank line followed by the raw image data.
You will need to ensure you are treating the image data as bytes and not encoding them while processing, otherwise you will get corrupt images
Pagination
GET /trestle/odata/Property?$top=1000&$skip=3000 HTTP/1.1
Host: api-trestle.corelogic.com
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1Ni...3wQJVBA
public static async Task Main(string[] args)
{
Client.BaseAddress = new Uri("https://api-trestle.corelogic.com/trestle/odata/");
await Authenticate(clientId, clientSecret);
var jsonDocument = await GetData("Property", "$top=1000&$skip=1000");
}
import org.apache.olingo.client.api.ODataClient;
import org.apache.olingo.client.api.communication.request.retrieve.ODataEntitySetIteratorRequest;
import org.apache.olingo.client.api.domain.ClientEntity;
import org.apache.olingo.client.api.domain.ClientEntitySet;
import java.net.URI;
String service_uri = "https://api-trestle.corelogic.com/trestle/odata/";
ODataClient oDataClient = ODataClientFactory.getClient();
URI uri = oDataClient.newURIBuilder(service_uri)
.appendEntitySetSegment("Property")
.top(1000)
.skip(1000)
.build();
ODataEntitySetIteratorRequest<ClientEntitySet, ClientEntity> request =
oDataClient.getRetrieveRequestFactory().getEntitySetIteratorRequest(uri);
import (
"encoding/json"
"golang.org/x/oauth2"
"golang.org/x/oauth2/clientcredentials"
"io/ioutil"
)
client := conf.Client(context.Background())
resp, _ := client.Get(base_uri + "/Property?$top=1000&$skip=1000")
var r OdataResponse
data, _ := ioutil.ReadAll(resp.Body)
_ = json.Unmarshal(data, &r)
let response = await trestle
.get("Property")
.query({$top: 1000, $skip: 1000})
.catch(e => console.log(e));
console.log(response);
r = session.get(f"{base_uri}/Property?$top=1000&$skip=1000")
properties = r.json()['value']
extern crate reqwest;
extern crate tokio;
extern crate serde_json;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let token = get_raw_token(CLIENT_ID, CLIENT_SECRET, TOKEN_URI, "api").await?;
let mut auth_header : String = "Bearer ".to_owned();
auth_header.push_str(token.as_str());
let client = reqwest::Client::new();
let resp = client.get("https://api-trestle.corelogic.com/trestle/odata/Property")
.query(&[("$skip", 1000), ("$top", 1000)])
.header("Authorization", auth_header)
.send().await?
.text().await?;
let v : serde_json::Value = serde_json::from_str(&resp).unwrap();
println!("{}", v["value"][0]["ListingKey"].as_str().unwrap());
Ok(())
}
Now that we are successfully downloading some property records and some associated images, it is time to download more. If you did not play with the $top
parameter yet, you likely noticed that you were only getting 10 records at a time. While this is great for getting started, it is not useful for a production system. So our first step will be to increase the number of records we get per query.
To get more records per query, we use the $top
parameter. If you want just one record, you can specify $top=1
or if you want one hundred, $top=100
. Trestle has a maximum of 1000 records per query, so the highest you can go is $top=1000
.
There are certain queries that can return more. They are discussed in Growing to Scale
As you can imagine, it is likely that you will be dealing with more than 1,000 records, so you will need to page through the results to get all of them. So, if your first query was /trestle/odata/Property?$top=1000
, your second query will be /trestle/odata/Property?$top=1000&$skip=1000
. We are adding the $skip
parameter to tell Trestle to skip the first 1,000 records.
As mentioned in the Fetching Property Data section, Trestle returns a JSON object with a field called @odata.nextLink
. You can use this value to automate your pagination for you, so that you don't have to keep track of what your $skip
value should be. Again, if your first query was /trestle/odata/Property?$top=1000
, Trestle would return in the JSON object '@odata.nextLink': 'https://api-trestle.corelogic.com/trestle/odata/Property?$top=1000&$skip=1000'
.
For replicating large numbers of records, use the Replication Endpoint
Quotas and Limits
HTTP/1.1 200 OK
Date: Wed, 25 Mar 2020 19:05:04 GMT
Content-Type: application/json; odata.metadata=minimal
...
QuotaType: OAuth
Minute-Quota-Limit: 4920.0
Hour-Quota-Limit: 147600.0
Hour-Quota-ResetTime: 1585163345000
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text.Json;
using System.Threading.Tasks;
namespace QuotaTool
{
internal class Program
{
private const string ClientId = "client_id";
private const string ClientSecret = "client_secret";
public static async Task Main(string[] args)
{
var client = new HttpClient();
var stringContent = new FormUrlEncodedContent(new[]
{
new KeyValuePair<string, string>("client_id", ClientId),
new KeyValuePair<string, string>("client_secret", ClientSecret),
new KeyValuePair<string, string>("grant_type", "client_credentials"),
new KeyValuePair<string, string>("scope", "api"),
});
var tokResp = await client.PostAsync("https://api-trestle.corelogic.com/trestle/oidc/connect/token",
stringContent);
var tokBody = await tokResp.Content.ReadAsStringAsync();
var tokJson = JsonDocument.Parse(tokBody);
var token = tokJson.RootElement.GetProperty("access_token").GetString();
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token);
client.DefaultRequestHeaders.UserAgent.ParseAdd("TrestleQuotaTool/1.0 CSharp");
var quotaResp =
await client.GetAsync(
"https://api-trestle.corelogic.com/trestle/odata/Media?$select=MediaURL&$top=1");
var apiMql = float.Parse(quotaResp.Headers.GetValues("Minute-Quota-Limit").First());
var apiHql = float.Parse(quotaResp.Headers.GetValues("Hour-Quota-Limit").First());
var quotaBody = await quotaResp.Content.ReadAsStringAsync();
var quotaJson = JsonDocument.Parse(quotaBody);
var mediaUrl = quotaJson.RootElement.GetProperty("value")[0].GetProperty("MediaURL").GetString();
client.DefaultRequestHeaders.Authorization = null;
var mediaResp = await client.GetAsync(mediaUrl);
var mediaMql = float.Parse(mediaResp.Headers.GetValues("Minute-Quota-Limit").First());
var mediaHql = float.Parse(mediaResp.Headers.GetValues("Hour-Quota-Limit").First());
Console.Write($"Quotas for {ClientId}:\n\n");
Console.WriteLine("WebAPI Queries:");
Console.WriteLine($" * {apiMql:N0} per minute");
Console.WriteLine($" * {apiHql:N0} per hour");
Console.Write("\nPublic MediaURL Requests:\n");
Console.WriteLine($" * {mediaMql:N0} per minute");
Console.WriteLine($" * {mediaHql:N0} per hour");
}
}
}
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"net/url"
"strconv"
)
func main() {
clientId := "client_id"
clientSecret := "client_secret"
client := &http.Client{}
tokResp, _ := client.PostForm("https://api-trestle.corelogic.com/trestle/oidc/connect/token",
url.Values{
"client_id": {clientId},
"client_secret": {clientSecret},
"grant_type": {"client_credentials"},
"scope": {"api"},
})
tokText, _ := ioutil.ReadAll(tokResp.Body)
var tokMap map[string]string
_ = json.Unmarshal(tokText, &tokMap)
tok := tokMap["access_token"]
quotaReq, _ := http.NewRequest("GET", "https://api-trestle.corelogic.com/trestle/odata/Media?$select=MediaURL&$top=1", nil)
quotaReq.Header.Add("Authorization", "Bearer " + tok)
quotaReq.Header.Add("User-Agent", "TrestleQuotaTool/1.0 Go")
quotaResp, _ := client.Do(quotaReq)
apiMql, _ := strconv.ParseFloat(quotaResp.Header.Get("Minute-Quota-Limit"), 64)
apiHql, _ := strconv.ParseFloat(quotaResp.Header.Get("Hour-Quota-Limit"), 64)
qText, _ := ioutil.ReadAll(quotaResp.Body)
var qMap map[string]interface{}
_ = json.Unmarshal(qText, &qMap)
mediaUrl := qMap["value"].([]interface{})[0].(map[string]interface{})["MediaURL"].(string)
mediaResp, _ := client.Get(mediaUrl)
mediaMql, _ := strconv.ParseFloat(mediaResp.Header.Get("Minute-Quota-Limit"), 64)
mediaHql, _ := strconv.ParseFloat(mediaResp.Header.Get("Hour-Quota-Limit"), 64)
fmt.Printf("Quotas for %s:\n\n", clientId)
fmt.Println("WebAPI Queries:")
fmt.Printf(" * %s per minute\n", strconv.FormatFloat(apiMql, 'f', 0, 64))
fmt.Printf(" * %s per hour\n\n", strconv.FormatFloat(apiHql, 'f', 0, 64))
fmt.Println("Public MediaURL Requests:")
fmt.Printf(" * %s per minute\n", strconv.FormatFloat(mediaMql, 'f', 0, 64))
fmt.Printf(" * %s per hour\n", strconv.FormatFloat(mediaHql, 'f', 0, 64))
}
import com.google.gson.Gson;
import com.google.gson.JsonElement;
import com.google.gson.JsonParser;
import org.apache.http.Consts;
import org.apache.http.NameValuePair;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.message.BasicNameValuePair;
import java.io.IOException;
import java.util.*;
public class main {
private static final String ClientId = "client_id";
private static final String ClientSecret = "client_secret";
public static void main(String[] args) throws IOException {
CloseableHttpClient client = HttpClients.createDefault();
List<NameValuePair> tokenParams = new ArrayList<>();
tokenParams.add(new BasicNameValuePair("client_id", ClientId));
tokenParams.add(new BasicNameValuePair("client_secret", ClientSecret));
tokenParams.add(new BasicNameValuePair("grant_type", "client_credentials"));
tokenParams.add(new BasicNameValuePair("scope", "api"));
UrlEncodedFormEntity entity = new UrlEncodedFormEntity(tokenParams, Consts.UTF_8);
HttpPost httpPost = new HttpPost("https://api-trestle.corelogic.com/trestle/oidc/connect/token");
httpPost.setEntity(entity);
CloseableHttpResponse tokenResp = client.execute(httpPost);
String tokenBody = new String(tokenResp.getEntity().getContent().readAllBytes());
tokenResp.close();
Map tokenMap = new Gson().fromJson(tokenBody, Map.class);
HttpGet quotaGet = new HttpGet("https://api-trestle.corelogic.com/trestle/odata/Media?$select=MediaURL&$top=1");
quotaGet.addHeader("Authorization", "Bearer " + tokenMap.get("access_token"));
quotaGet.addHeader("User-Agent", "TrestleQuotaTool/1.0 Java");
CloseableHttpResponse quotaResp = client.execute(quotaGet);
String quotaBody = new String(quotaResp.getEntity().getContent().readAllBytes());
float apiMql = Float.parseFloat(quotaResp.getFirstHeader("Minute-Quota-Limit").getValue());
float apiHql = Float.parseFloat(quotaResp.getFirstHeader("Hour-Quota-Limit").getValue());
quotaResp.close();
JsonElement quotaRoot = JsonParser.parseString(quotaBody);
String mediaUrl = quotaRoot.getAsJsonObject()
.get("value").getAsJsonArray()
.get(0).getAsJsonObject()
.get("MediaURL").getAsString();
HttpGet mediaGet = new HttpGet(mediaUrl);
CloseableHttpResponse mediaResp = client.execute(mediaGet);
float mediaMql = Float.parseFloat(mediaResp.getFirstHeader("Minute-Quota-Limit").getValue());
float mediaHql = Float.parseFloat(mediaResp.getFirstHeader("Hour-Quota-Limit").getValue());
mediaResp.close();
System.out.println("Quotas for " + ClientId);
System.out.print("\nWebApi Queries:\n");
System.out.printf(" * %,.0f per minute\n", apiMql);
System.out.printf(" * %,.0f per hour\n", apiHql);
System.out.print("\nPublic MediaURL Requests:\n");
System.out.printf(" * %,.0f per minute\n", mediaMql);
System.out.printf(" * %,.0f per hour\n", mediaHql);
}
}
const {URLSearchParams} = require('url');
const fetch = require('node-fetch');
const CLIENT_ID = "client_id";
const CLIENT_SECRET = "client_secret";
async function run() {
const params = new URLSearchParams();
params.append("client_id", CLIENT_ID);
params.append("client_secret", CLIENT_SECRET);
params.append("grant_type", "client_credentials");
params.append("scope", "api");
const tokenResp = await fetch("https://api-trestle.corelogic.com/trestle/oidc/connect/token", {
method: "POST",
body: params
});
const token = await tokenResp.json();
const quotaResp = await fetch("https://api-trestle.corelogic.com/trestle/odata/Media?$select=MediaURL&$top=1", {
headers: {'Authorization': 'Bearer ' + token['access_token'],
'User-Agent': 'TestleQuotaTool/1.0 Javascript'}
});
const quotaJson = await quotaResp.json();
const apiMql = Math.floor(parseFloat(quotaResp.headers.get('Minute-Quota-Limit')));
const apiHql = Math.floor(parseFloat(quotaResp.headers.get('Hour-Quota-Limit')));
const mediaResp = await fetch(quotaJson['value'][0]['MediaURL']);
const mediaMql = Math.floor(parseFloat(mediaResp.headers.get('Minute-Quota-Limit')));
const mediaHql = Math.floor(parseFloat(mediaResp.headers.get('Hour-Quota-Limit')));
console.log("Quotas for " + CLIENT_ID + "\n");
console.log("WebAPI Queries:");
console.log(" * " + apiMql + " per minute");
console.log(" * " + apiHql + " per hour");
console.log("\nPublic MediaURL Requests:");
console.log(" * " + mediaMql + " per minute");
console.log(" * " + mediaHql + " per hour");
}
run().then(null);
import requests
if __name__ == '__main__':
client_id = 'client_id'
client_secret = 'client_secret'
tok_resp = requests.post('https://api-trestle.corelogic.com/trestle/oidc/connect/token', data={
'client_id': client_id,
'client_secret': client_secret,
'grant_type': 'client_credentials',
'scope': 'api'
})
quota_resp = requests.get('https://api-trestle.corelogic.com/trestle/odata/Media?$select=MediaURL&$top=1',
headers={'Authorization': 'Bearer ' + tok_resp.json()['access_token'],
'User-Agent': 'TrestleQuotaTool/1.0 Python'})
api_mql = float(quota_resp.headers['Minute-Quota-Limit'])
api_hql = float(quota_resp.headers['Hour-Quota-Limit'])
media_resp = requests.get(quota_resp.json()['value'][0]['MediaURL'])
media_mql = float(media_resp.headers['Minute-Quota-Limit'])
media_hql = float(media_resp.headers['Hour-Quota-Limit'])
print(f"""Quotas for {client_id}:
WebAPI Queries:
* {api_mql:,.0f} per minute
* {api_hql:,.0f} per hour
Public MediaURL Requests:
* {media_mql:,.0f} per minute
* {media_hql:,.0f} per hour
""")
extern crate reqwest;
extern crate tokio;
extern crate serde_json;
use std::collections::HashMap;
use num_format::{Locale, ToFormattedString};
const CLIENT_ID : &str = "client_id";
const CLIENT_SECRET : &str = "client_secret";
const TOKEN_URI : &str = "https://api-trestle.corelogic.com/trestle/oidc/connect/token";
pub async fn get_raw_token(client_id: &str, client_secret: &str, token_uri: &str, scope: &str)
-> Result<String, reqwest::Error> {
let mut map = HashMap::new();
map.insert("client_id", client_id);
map.insert("client_secret", client_secret);
map.insert("grant_type", "client_credentials");
map.insert("scope", scope);
let client = reqwest::Client::new();
let body = client.post(token_uri)
.form(&map)
.send().await?
.text().await?;
let v : serde_json::Value = serde_json::from_str(&body).unwrap();
Ok(v["access_token"].as_str().unwrap().to_string())
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let token = get_raw_token(CLIENT_ID, CLIENT_SECRET, TOKEN_URI, "api").await?;
let mut auth_header : String = "Bearer ".to_owned();
auth_header.push_str(token.as_str());
let client = reqwest::Client::new();
let resp = client.get("https://api-trestle.corelogic.com/trestle/odata/Media?$select=MediaURL&$top=1")
.header("Authorization", auth_header)
.send().await?;
let h = resp.headers();
let api_mql = h.get("minute-quota-limit").unwrap().to_str().unwrap().parse::<f32>().unwrap() as u64;
let api_hql = h.get("hour-quota-limit").unwrap().to_str().unwrap().parse::<f32>().unwrap() as u64;
let body = resp.text().await?;
let v : serde_json::Value = serde_json::from_str(&body).unwrap();
let media_url = v["value"][0]["MediaURL"].as_str().unwrap();
let resp = client.get(media_url)
.send().await?;
let h = resp.headers();
let media_mql = h.get("minute-quota-limit").unwrap().to_str().unwrap().parse::<f32>().unwrap() as u64;
let media_hql = h.get("hour-quota-limit").unwrap().to_str().unwrap().parse::<f32>().unwrap() as u64;
println!("Quotas for {}", CLIENT_ID);
print!("\nWebAPI Queries:\n");
println!(" * {} per minute", api_mql.to_formatted_string(&Locale::en));
println!(" * {} per hour", api_hql.to_formatted_string(&Locale::en));
print!("\nPublic MediaURL Requests:\n");
println!(" * {} per minute", media_mql.to_formatted_string(&Locale::en));
println!(" * {} per hour", media_hql.to_formatted_string(&Locale::en));
Ok(())
}
Now that we have our queries set up to download 1,000 records at a time, we are ready to open the flood gates and get all of the property records, all of the media, all of the member data, etc. Well, not so fast. Literally, not so fast. Trestle has a quota system such that you can only make so many queries per hour or per minute.
For your connection, you have two sets of quotas. One for API queries and one for Public URL requests (the URLs you got from querying the Media resource). Each set of quotas has a per hour quota and a per minute quota. The per hour quota is your baseline quota. It is the number of queries you can make per hour and what you should be aiming for when developing your application. The per minute quota is a burst quota.
You are probably wondering at this point what your quotas are. They are calculated per product/feed type pair. So, if you have a product Jim's Pretty Good Real Estate Website and it has an IDX - WebAPI connection to one MLS and an IDX Plus - WebAPI connection to another MLS, then each one of those connections will get its own quota. If, you add three more MLSs to the IDX - WebAPI connection, then that connection will get more quota.
As part of the HTTP response to an API request, Trestle returns three headers that tell you about your quotas.
Minute-Quota-Limit
specifies the number of queries per minute (your burst quota) you can makeHour-Quota-Limit
specifies the number of queries per hour you can makeHour-Quota-ResetTime
specifies when the per-hour quota will reset as a Unix timestamp in milliseconds (1585163345000
is equivalent to Wednesday, March 25, 2020 7:09:05 PM GMT)
The baseline quotas are 7,200 queries per hour and 180 queries per minute for WebAPI queries and 18,000 requests per hour and 480 requests per minute for public media URLs.
You can use one of the provided code samples in this section to extract your WebAPI quotas from Trestle. These scripts will count as quota hits, so while they can be a useful tool for getting the information, we do not recommend running them very often.
Errors
Error Code | Error Name | Description |
---|---|---|
400 | Bad Request | Indicates an invalid set of credentials or connection type |
401 | Unauthorized | Indicates an attempt to access the WebAPI without a valid access token |
404 | Not Found | Indicates an attempt to access a resource that does not exist or is not available |
429 | Quota Exceeded | Indicates too many attempts in the past minute or hour |
500 | Server Error | Indicates an error in Trestle |
504 | Gateway Timeout | Indicates that the query took too long to process |