How to Screen Scrape Salesforce with REST GET in Apex?
Question:
How can you make a REST GET call from Apex to retrieve data from Salesforce, particularly from standard pages like /001/o or /home/home.jsp? When using the standard Salesforce REST API, the request works for objects (e.g., /services/data/v26.0/sobjects/User/describe), but trying to scrape a regular page results in receiving a placeholder HTML response. How can you access the actual content of these pages?
How can you make a REST GET call from Apex to retrieve data from Salesforce using an endpoint like /services/data/v26.0/sobjects/User/describe? In one example, the following code works as expected: What can be done to get the actual content of a Salesforce page using a REST GET request?
Answer:
To make a REST GET request in Apex, you can use the Http and HttpRequest classes to send the request to Salesforce’s REST API. However, there is a difference in how you need to authenticate depending on whether you’re working with standard Salesforce object data or a regular page like /001/o or /home/home.jsp.
Boost your Salesforce career with CRS Info Solutions expert-led Salesforce training, hands-on projects, and a free demo—perfect for beginners and professionals!!!
For API calls to Salesforce object data, you can use the session ID in the Authorization header like this:
String requestUrl = '/services/data/v26.0/sobjects/User/describe';
Http http = new Http();
HttpRequest req = new HttpRequest();
req.setEndpoint(URL.getSalesforceBaseUrl().toExternalForm() + requestUrl);
req.setMethod('GET');
req.setHeader('Authorization', 'Bearer ' + UserInfo.getSessionId());
HTTPResponse res = http.send(req);
String output = res.getBody();
System.debug(output);Explanation:
The code sends a GET request to Salesforce’s REST API to retrieve metadata about the User object using the describe endpoint. It constructs the request URL, sets the authorization header with the session ID, and sends the request via the Http class. The response body, which contains the metadata, is captured and logged for debugging purposes.
This works fine for retrieving object data, as the Authorization header is appropriate for API requests. However, when attempting to scrape a standard Salesforce page, like /001/o, you encounter a redirect or placeholder HTML instead of the actual page content.
To properly access standard Salesforce pages, you need to send the session ID as a cookie, rather than using the Authorization header. The correct approach would look like this:
String requestUrl = '/001/o'; // Standard page URL
Http http = new Http();
HttpRequest req = new HttpRequest();
req.setEndpoint(URL.getSalesforceBaseUrl().toExternalForm() + requestUrl);
req.setMethod('GET');
req.setHeader('Cookie', 'sid=' + UserInfo.getSessionId());
HTTPResponse res = http.send(req);
String output = res.getBody();
System.debug(output);Explanation:
The code sends a REST GET request to a standard Salesforce page (e.g., /001/o) using the Http and HttpRequest classes in Apex. It sets the Cookie header with the session ID (sid), allowing Salesforce to recognize the session and return the actual page content. The response body is captured and logged using System.debug().
This approach ensures that Salesforce recognizes the session and delivers the actual page content. The response will include the full HTML data for the page, rather than a placeholder or redirect.
Additional Notes:
While screen scraping may seem like a quick solution, it’s generally fragile and not the best approach for integrations. With the introduction of the Analytics API, there are more reliable and efficient ways to retrieve data from Salesforce. Using cookies to authenticate for page scraping can work, but it’s not ideal for long-term solutions.
Unlock Your Future with Salesforce Training in Chicago
Transform your career with our premier Salesforce training in Chicago, designed to fast-track your professional growth. Whether you’re a beginner or an experienced professional, our comprehensive course offers expert-level instruction in Salesforce CRM, hands-on projects, and guidance to help you earn certifications like Salesforce Administrator and Developer.
Our Salesforce training focuses on industry-relevant skills, with personalized mentorship, detailed course materials, and continuous support throughout certification and interview prep. Whether you’re starting fresh or looking to enhance your expertise, our seasoned instructors are committed to helping you succeed in a competitive job market.
Take the first step towards an exciting Salesforce career—join our free demo session today!!!
Related Posts:
