[deleted]
Before getting started, and I'm sorry if this comes across as nitpicky here, but please don't use percentage and mg/ml of nicotine interchangeably. Nicotine has a density of approximately 1000mg/ml, so your 100mg/ml base is 10% nicotine; your post indicates you're going for 3mg/ml, so while you would want that nic base to be 3% of your total mix, that's 10% times 3%, or 0.3% - saying "3% nic" implies 30mg/ml.
Regarding the weight calculations, you used 1.036g/ml density for your nic, so I'm assuming it's PG based? If so, 90% PG plus 10% nic would be 1.036*0.9+0.1=1.032g/ml, or 15.49g. That said, for a mix of this size that's not a significant difference, I wouldn't sweat it.
As for what you did measuring manually by volume, it's not ideal, but if it works it works. In the future I would advise weighing the nicotine first, then the PG, then the VG, so your margin of error increases as you go.
Also, I'm curious what scale you're using: you saying your scale reset and wouldn't register any weight suggests you might have overloaded it, you're indicating a 0.01g resolution going close to 1kg total including the bottle, but it only has a single 50g calibration weight? For comparison's sake, my usual scale for mixing is an old LB501, which has two 200g calibration weights, precision to 0.01g, but its capacity is only 500g so the biggest mixes I do on it are ~240ml, anything bigger than that I mix just the nic+flavors (and maybe PG) on the LB501 then finish with the diluent on a kitchen scale (1g up to 10kg, hence the mixing order from lowest to highest margin of error).
Finally, regarding your mix not being filled to the brim when it was previously, there's a possibility you didn't mention: the bottles you were buying were overfilled. For how big the difference is between wholesasle cost of ingredients versus sale price of the finished product, the math could easily work out that the time investment needed to get exactly 500ml per bottle costs more than any loss from "just mix a huge-ass batch and fill up the bottles". Of course it's hard to say without knowing the difference in volume - "like an inch" doesn't really say much if we don't know the dimensions of the bottle in question, y'know?
One issue you might be having - you overloaded your scale. The most common scale used in DIY is the AWS LB-501 and it has a max capacity of 500g. If you're mixing 500ml and most of the mix is VG (SG=1.26g.ml) you're already over 500g. And then you have to add in the weight of the bottle you're mixing in. In the future you should shoot for approx 400ml total so you don't overload your scale. If your scale can handle more - disregard everything I just said.