I recently added a test function to check various vendors for flavor concentrate stock status. It took all of 2 minutes to add to a single page, so obviously it isn't a lot of work (until you consider 500 flavor pages lol).
I'm curious if any website already offers this service?
Right now I am only checking one flavor from ecx, bcv, rts, nicotine river and gremlindiy, but I figure it won't be hard to pull the status data from other vendors automatically.
But if there's already a website that offers it, no point in my reinventing the wheel.
While we're on the subject, I'm super excited to let everyone know that we are finally getting nice and saturated. The past while has been a battle to keep product in stock. Orders keep going through the roof! We've recently made a multitude of upgrades, and I'm excited to see the improvement!
Demand is up from mixers and juice sellers, so the distributors put a lot of pressure on the compounders to mix more!
I bet you'll need a bigger warehouse soon.
That would be a pretty smart bet to make. We are beginning construction shortly.
Haven't heard of anything like this but it would definitely come in handy... maybe an addition to your vape mail site???
It's a different site completely. Mostly for my own information gathering purposes but it's currently public facing until I decide if I want to try to monetize it to keep it public or just make it password/invite only for those I like.
this is an amazing idea!
I'm a little bit surprised it isn't done anyway else. Since it only took me 2 minutes, I figure I might as well just keep it -- it only requires copying and pasting URLs into a function I wrote. But I have to do that on every flavor page, woof.
Make another page to crowdsource the work. I'd be willing to help as I'm sure many of us would.
If I monetize it, I don't want any volunteers coming after me for a dime. I did that a decade ago with a site I sold 3 months later for a decent price but nothing to write home about, and 20 "friends" all came after me for more than the buyer was willing to pay them for their contributions.
I also don't like volunteer work if I can avoid it. I'd rather pay 2 people to do work than get help from 20 people gratis.
Please refer to the sidebar or use the search tool before reposting questions like this, then check here
Are you doing this by scraping or API?
Scraping -- I don't see many APIs available. I'd limit scrapes to every 4 hours probably and just cache it for 4 hours.
The function I wrote is easy peasy scraping and appears to work on 6 vendors' sites.
Well yeah scraping is easy peasy but can be hell on a server (even if it's not if their hosting provider decides to lock out the IP for "suspicious activity" then your scraper goes down). Ideally you should be able to contact each vendor and have them generate an API key for you to make better work of it all.
From what I can see here's the info you need
Vendor | eCommerce Platform | API Documentation | Contact ---|---|----|---- ECX | Magento | Magento API (Add-on installation needed) | Ticket System (Login Needed) BCV | BigCommerce | BigCommerce API | support@BullCityVapor.com RTSVapes.com | Volusion | Volusion API | Contact Form Nicotine River | Shopify | Shopify API | Info@nicotineriver.com GremlinDIY | BigCommerce| BigCommerce API| Contact Form
Should help you out a lot more than scraping and caching. What language are you developing in?
EDIT: Also EcigExpress looks like it will be the hardest/easiest to convince since they're so big.
Yep, if I decide to publicize it, I'm sure an API makes a lot more sense. Since it's personal use only right now, the scraping is no more aggressive than if my own browser was getting the data. Same data transferred. And if I don't pull data that's recently cached within 4 hours, the scraping won't occur until the next time I visit THAT flavor and the cache is expired.
No, but there's one that will within a few weeks. :)
It's on the timeline for all the flavors, but not done yet.
I wrote something similar a while back for trading cards. It was a bunch of shells that scraped different sites, grabbed case/hobby box pricing and dumped it all in a db and on the frontend it displayed cheapest to highest. It also kept pricing history and some other nifty stats.
It was made for my own use because i'm honestly not sure how legal scraping is.
I still have the code somewhere, it's not commented really but its pretty straight forward php mvc type stuff.
Yeah, once you have the data it's easy to present it in a useful fashion.
I'm adding code right now that let's me go to a flavor page and say "I need this" it it creates a "flavor cart". Then I can go to a page that says "match vendors" and it'll show me what the smallest number of shipments I can make is.
Not sure how well that code will work out but I figure it's just a matter of sorting, parsing, matching and filtering. Can't be THAT bad.
If I get that working, I also want to add the feature of what my concentrate bottles currently weigh -- since I know what that vendor's empty bottle weighs, it'll tell me when I am getting low, and give me the option to build a new flavor cart.
I'm sure /u/queuetue's awesome app does the same thing, or will do the same thing, but I prefer web apps to phone apps due to the bigger screen (touch screen on my notebook I lug around).
The first part of this is in place now - most of the linking is done. "In stock" status will follow shortly. https://alltheflavors.com/flavors/capella-golden-pineapple
The vendors that have the most flavorings in stock will most likely get the most orders .
I can't tell you how many times i have canceled going through with an order due to an item being out of stock.
Probably true but it's complicated in some ways because there's also the need to sort by pricing AND shipping costs.
- Vendor A has 1,2,3,4,5
- Vendor B has 3,4,5,6,7
- Vendor C has 1,3,5,7
So it's a bit of a pain to sort by price/availability/total. Lots and lots of recursive math.
This would save so much time shopping about 4 different websites or so get get the flavours I need. Any chance you could as UK vendors?