I have noticed a couple of issues that may or may not be related.
Issue 1: I have created a custom integration using the API to create items in Monday whenever a contract signed event webhook is fired from our proposal software. Most of the time it works fine. Occasionally though, the item will have blank entries in a few columns despite the API call returning a success code without an error message. To make it even more confusing, the activity log in Monday shows that the value was entered into the column, and no action appears after to indicate that it has been deleted or rolled back. To me this means there is either a bug that is preventing the item from displaying correctly, or the column value is being deleted without record of this happening in the activity log (undocumented roll back). If it is a roll back, this is a huge problem, because whatever is triggering the roll back is seemingly random, and is not consistent with any valid reason for rolling back an update (ie a conflict such as entering a column value into a deleted column).
Issue 2: Automations that trigger on item creation fail silently. I have an automation that assigns a value to the people column whenever an item is created. This works fine most of the time, but occasionally this automation will silently fail, and the people column will be left unpopulated.
If I had to guess, these two errors are similar, and potentially related to API rate limits or concurrency issues. Silently rolling back an update that is 100% not conflicting to a prior or future operation seems like completely unintended behavior.
Has anyone else noticed similar behavior and have any idea of what might be causing this, and or what I can do to work around this? (For the time being I solved issue 2 by setting default values for columns, but I am concerned that there are larger issues lurking below if a cookie cutter out of the box automation is failing silently.)
I would be happy to discuss this in more detail, or provide code snippets and or AWS / Monday activity logs if it will help narrow down the problem.
I experience similar behavior. We are running 10+ different apps and close to 1.000 different accounts are using these apps. I most cases everything works as expected. It is the last 4-5 weeks that I am receiving calls from customers that information written to their board is not visible. As I log each and every API call in a logger solution based on ELK, I have proof that a value is written to a board / item / column. The customer has the same proof from the activity log. Like you, for me that is ultimate proof that the information has been written to the board successfully. However, the customer(s) claim the information is not visible on their board. Off course this can be because the user (or another app / automation) has deleted it, but there is no sign at all in the activity log. It very much looks like a roll back.
I am redirecting these customers to support, but with very limited success. The support team states they can’t support the customer as it involves a 3rd party app. I also reached out to the appsupport team (mainly @dipro and @Matias.Monday) and hope they are willing to invest time to sort this out to the bottom. I am fine with giving access to my logger environment (if they know how to use Kabana / Lucene) as the logs shows both the actual API call, the variables and the success (to the millisecond).
I am very interested to work together with you with the end goal to make the API more robust.
I just had a similar experience in which the monday activity log showed that a value was written to a text column hours prior to accessing the same value using the API where it read as null. Subsequent reads showed the correct value. There were no other expected changes to the value and monday logs confirmed no other changes.
If we can not rely on data being accurately displayed and returned, all is lost.
To chip in here, monday.com has a concurrency problem. It is a problem I have mentioned before here. @dipro and @Matias.Monday, this is the problem I was talking about during the developers meeting but couldn’t remember exactly what it was.
The workaround currently for multiple values is to create each column value one after the other. This will of course take way longer time but at least, that is works for me for now
Thanks for the response. I would love to collaborate with you to find a solution to this problem. What time zone do you work in? Maybe we can set up a Zoom call one day next week to share thoughts and ideas. My fear is that this issue is fundamental to Monday’s backend architecture, and that we will be at the mercy of the Monday dev team for a solution. That said though, I’m definitely up for trying to find a temporary work around until they can get this fixed.
Thanks for the reply!
Unfortunately I don’t think that workaround will work for my use case. I am running an AWS Lambda function for the integration, and splitting all of the column value insertions into single API calls would almost certainly cause the function to timeout. (The function already runs for almost a minute and a half, and splitting up the calls would result in close to 10x as many API calls). I suppose I could break it up into multiple Lambda functions, but this is far from ideal and will result in significantly higher AWS bills. Hopefully Monday can figure out their concurrency issues in a timely manner.
Thanks for your response. It’s good to know that I’m not the only one experiencing these issues. I was starting to pull my hair out yesterday figuring out what was going on. Hopefully we can all get together some time next week to find a solution to this.
Exactly the problem I was facing. I have migrated to another service provider due to that reason alone. But yes, the costs is another issue you will have deal with due to the multiple requests
I am in CET (Rotterdam, The Netherlands). You can setup a call through Calendly - Bas de Bruin. I agree that this looks like an architectural issue, but in my experience this only happens since approx. 6 weeks. I am also working with the appsupport team and a customer who experiences missing values and I think it is good to align efforts.
If you are experiencing these kind of issues, you can always contact us at appsupport@monday.com and in this way we will have a great way of following the issue and updating you when we have updates about it!
Thank you for the response. It sounds like quite a few people are experiencing the same issue. If I can provide an assistance in this effort please let me know.
