Hello! Thanks for developing this tool. I'm trying to use it to automate visual regression tests for my static site, since it has lots of pages and I can't always keep an eye on everything. However, I'm unable to get anything working. I've set up backstop as per the documentation in the README, cross-checked with a "static site" example, and made adjustments that I thought would help -- all to no avail: I am consistently getting false positives.
Specifically, it seems like about 25% of all my pages do not load properly when either making references or comparing. One clear example of this is a page with default styling (evidently missing CSS), but there are other, more subtle problems (e.g., a page in which all text is italicised, for some reason?). There seems to be a fixed probability of this happening for any given page. As a result, after 15 runs, I haven't once had a clean report while creating a reference of a page, then comparing that same page, unchanged, to the reference.
I'm running backstop on a python -m http.server local server over HTTP. I have configured the load wait time to be 5 seconds (to give the chromium instance time to load all the assets etc). My site is less than 200kb total per page, so I can't imagine why it would even need that long. When I manually open the site in Chrome, and spam "Refresh without cache", there is visually no load time at all.
What am I doing wrong? Are the steps I can take to mitigate the weird loading failures? Is there something else I'm missing?
Thank you!
Example of a failure:
Hello! Thanks for developing this tool. I'm trying to use it to automate visual regression tests for my static site, since it has lots of pages and I can't always keep an eye on everything. However, I'm unable to get anything working. I've set up backstop as per the documentation in the README, cross-checked with a "static site" example, and made adjustments that I thought would help -- all to no avail: I am consistently getting false positives.
Specifically, it seems like about 25% of all my pages do not load properly when either making references or comparing. One clear example of this is a page with default styling (evidently missing CSS), but there are other, more subtle problems (e.g., a page in which all text is italicised, for some reason?). There seems to be a fixed probability of this happening for any given page. As a result, after 15 runs, I haven't once had a clean report while creating a reference of a page, then comparing that same page, unchanged, to the reference.
I'm running backstop on a
python -m http.serverlocal server over HTTP. I have configured the load wait time to be 5 seconds (to give the chromium instance time to load all the assets etc). My site is less than 200kb total per page, so I can't imagine why it would even need that long. When I manually open the site in Chrome, and spam "Refresh without cache", there is visually no load time at all.What am I doing wrong? Are the steps I can take to mitigate the weird loading failures? Is there something else I'm missing?
Thank you!
Example of a failure: