Importing Multiple Boards to Databricks

Hi, Community.

I’m currently trying to import the data of some boards into databricks. All of them have the same structure, and I wanted to get all the columns (if possible).

Is there a way to make only one request and get the data of all the boards?

I’ll put down here the script of the first step, which is the one we define the boards’ ID.

%python

apiKey = "censored" ## API Key

apiUrl = "https://api.monday.com/v2"
headers = {"Authorization" : apiKey}

query = '''query {
 boards(ids:[1272092066,1615564124]) {
  items {
    id
    name
    column_values {
      title
      text
} } } }'''
data = {'query' : query}

r = requests.post(url=apiUrl, json=data, headers=headers) # make request
data = r.json()

I tried putting the IDs separated by commas, but at the end it only got the data of one of these boards.

Am I missing something? To be honest, I’ve never worked with APIs, so all of this is really new to me. I couldn’t find a part on the documentation that specified this type of integration.

If anyone that’s willing to help need more informations, just let me know!

Thanks a lot!

Hey @renan.crepaldi :wave:

Sorry for the delay here! It seems like your section was in the section for native monday.com Automations and Integrations. The best place to ask questions related to using the monday.com API and Apps Framework is the Monday apps & Developers section :slight_smile:

To be transparent with you, to me, your query seems correct and it should return the data of 2 different boards. Is there any chance the data output contains the info of the second board further in the data you receive?

As for your questions here, I wouldn’t recommend querying the data of all boards and their column values in a single query. Instead, it would be best to query those boards in small, lightweight batches and run multiple focused queries instead. That way, you will avoid running into issues related to timeouts or complexity-related errors.

I’d love to troubleshoot this further over email, so I’m going to follow up there with further questions to investigate this one. :slight_smile:

-Alex

1 Like

Hello, Alex.

Thanks for answering! I’ve been trying to solve this problem since the day I posted it, and I think the problem is not on this first query indeed.

Just to clarify, I’ve never used APIs as an origin on Databricks, so all of this is really new to me. The queries I’m using were made by a member of our team for another project, and I was trying to adapt them.

Is it possible for you to show me what the ideal structure should be? I wanted to import all the columns of a group of boards (approximately 10).

I read that for importing multiple boards the ideal should use a loop, right?

Thanks again!

@renan.crepaldi

Thank you for following up on this!

Just to confirm, if you make the API call directly via the API playground in your account, or via Postman, do you get the data of both the boards you specified via the ID?

In general, I do recommend going through the boards to get their columns in a loop. This way, you’ll ensure your queries are lightweight and low-complexity. This can help you avoid issues with timeouts. If you query 10 boards at once, and they are quite large with a high number of coolumns, this will likely take longer than 60 seconds to return the data and your call could be timed out.

I also recommend making the calls in an async way. In JavaScript, you could use a for await ... of loop, as this would only attempt to query the next board once you get the data of the previous board you’ve queried. If you query all boards at once, this could lead to 500 errors due to race conditions.

Let me know if this helps clarify!

-Alex