KerbalX.com - Craft & Mission Sharing

[swjr-swis]

4 hours ago, katateochi said:

 fixed that issue with the lists. 

Confirmed on Firefox 44, all numbers show without pixels lost.

[AVOS]

How to delete a craft?

[katateochi]

As you guys suggested storing multiple image sizes is the way to go, but wow has this been tricky!

On the off chance you want to read about my coder conundrums;

Spoiler

 

The problem with doing this is needing to process and then upload the 3 different sizes (:small, :medium and :wide) and that process can take 15 seconds or more.  That could easily hang the site if even just 2 users changed thumbnail at around the same time.  The other issue I had was the image library I'm using (paperclip, which uses imagemagick) absolutely insists on storing the "original " image too.  With my first attempt at this I got around that by just redefining "original" to be the dimensions of the small thumb, but with multi images this didn't work because it processes 'original' first and then uses that to process all the others (so they all ended up really low quality).  

So the solution; When you first set an image on a craft, or when you change the thumbnail it very quickly processes just the small thumbnail, but treats it as the 'original' so only one small image is processed and uploaded.  So that request is nice and quick.  But what about the others? When the first image is processed it also queues up a job which will process the other sizes, but on the background worker queue so it can take as long as it likes without slowing the site down.  BUT this process has to change the definition of original back to a higher quality size so it can process the :medium and :wide thumbs from a higher res source (and then it will insist on uploading that, which would overwrite the already uploaded small thumb).  Thankfully I found how to change how it refers to 'original' so I could give it a slightly different name in the upload.  

So for each craft you wind up with four images, ie for craft with id 4242 you get 4242.jpg (which is the small thumb), 4242_medim.jpg, 4242_wide.jpg and 4242_original.jpg, but the first (4242.jpg) is uploaded on it's own and quickly (by tricking it into thinking that it is the original) and the others get uploaded a bit later and the bulk of the work is done on the worker queue.

This does mean that when you change a thumbnail or set one for the first time, the craft won't have it's higher res thumbnails so will use the lower res one until they've been processed, but that's fine.

 

The other thing that muggins here hadn't got right on my first go was setting the image storage up so that it could work with my CDN.  So up till now, images have been getting served directly out of the AWS storage (which has been racking up the number of GET requests substantially).  I've now figured out how to set the image storage up correctly (I think) so it will take advantage of the CDN's caching and it also makes for nicer urls.  Before the image urls were something like s3.amazonaws.com/thumbs.kx/10138.jpg now they are t.kerbalx.com/10138.jpg 

 

Anyway, the upshot of all that is multiple sized thumbnails are processed and stored without adding a slowdown to the site and that plus a proper CDN setup should mean that pages with lots of craft thumbnails should load up really quite quickly (faster than it has been this week and MUCH faster than it was when the images were hosted on imgur, *smug-face).  
I think there are still further gains to get out of tweaking the CDN, but I got some more learnin' to do! 

One thing I've not changed yet is where forum signatures are being hosted from.  That will happen soon and I'll post about that.  Going to need people who have the sigs to change the url that they are pointing to. I guess I'll just keep an eye out for people with them and send PMs.

Still no sign of activity from imgur.  I had one email back last week from a different support person who started with an apology for the delay in the reply and quickly and helpfully answered my questions and ended with saying that they hoped to hear back from me so they could re-evaluate the site and life the block.  I emailed back and ...........
The other rather odd thing is that the first support thread I opened when they said I'd been blocked, in which I detailed what I was doing to fix the problem (which at that point was barking up the wrong tree) and asked if that was correct, was never answered and has now been deleted.  It was still an open request, but it's been removed from my support account without a trace. interesting. I am getting the feeling that an unfortunate dice roll has assigned me someone who doesn't care.  So I guess time to write to them again.......joyous. 

Edited February 22, 2016 by katateochi

[swjr-swis]

Good work. Learning new tricks to get things done is my preferred way to gain new knowledge.

As for imgur: hard to do, but I advice patience dealing with them, not simply in time but in how you communicate with them. I use them because their way of image hosting suits my purpose and my way of working, but support-wise... it requires patience and calm to deal with them.

[katateochi]

1 hour ago, AVOS said:

How to delete a craft?

click on your craft to view it, and then in the grey header bar at the top there will be a "delete craft" link.

[katateochi]

20 minutes ago, swjr-swis said:

Good work. Learning new tricks to get things done is my preferred way to gain new knowledge.

As for imgur: hard to do, but I advice patience dealing with them, not simply in time but in how you communicate with them. I use them because their way of image hosting suits my purpose and my way of working, but support-wise... it requires patience and calm to deal with them.

Yeah I'll wait a bit more and see, waited over a week last time! And yes, I've kept it very polite and friendly with them (I just rant about them here, cos, well, need to let it out, soz!).  It is interesting how a company's level of support totally changes how you feel about them and in the last 2 weeks I've had support dealings with Razor (my mouse died), Heroku, Imgur, CloudFlare and NewRelic, and aside from imgur I've been v impressed with the quality of support from those (Razer in particular and CloudFlare for replying almost instantly, on a Sunday, to a non-paying customer!).  Imgur, imgur I want to shake (but I shall exercise restraint!).  Perhaps I've been spoilt by my dealings with those other companies!

[StormKat]

Katateochi, KerbalX won't let me login anymore.

I went through the "Forgot my password" link and I'm back in.  So weird.

Edited February 23, 2016 by StormKat

[katateochi]

14 minutes ago, StormKat said:

Katateochi, KerbalX won't let me login anymore.

I went through the "Forgot my password" link and I'm back in.  So weird.

hmmm that is odd. A LOT of behind scenes stuff has changed recently, but nothing that should affect that, the only thing I can think is that the new caching was handing you an old copy of a javascript asset or something. 

Now you've reset your password, could you try logging out, clearing your browser's cache (or if that's a hastle, open up an incognito session ctrl+shift+n on chrome) and try and login again and let me know if that gives you any problems. 

[StormKat]

That worked just fine.  Weird glitches like this catch me off guard.

[katateochi]

4 minutes ago, StormKat said:

That worked just fine.  Weird glitches like this catch me off guard.

yeah that sorta thing bugs me too.  I really dislike problems that just appear to fix themselves! 

[Spyke]

Hi, is it possible to update the .craft file of a posted craft? Thanks!

[katateochi]

22 minutes ago, Spyke said:

Hi, is it possible to update the .craft file of a posted craft? Thanks!

It is!  
- If the updated craft has exactly the same name as the one you've already uploaded then just upload it again like normal and you'll be asked if you want to update the existing one or create a new one.  
- If the updated craft has a different name to the one you want to update, then click on your craft and then at the top of the page click "Edit Craft".  Then upload the updated version. You will be asked to confirm that you want to update the current craft.  
Updating a craft will leave the page and pictures etc that you've added as they are, it just updates the file and info about the craft.

[katateochi]

Update on the Imgur situation
I have finally had a reply back from imgur and they are going to lift the block on KerbalX! (that process may take 24 hours).
Wooooo! 

 

I've also got a bunch of fixes ready to go up, but Heroku is doing some maintenance to it's developer tools so I'm going to wait till tomorrow.
Fixes include some improvements to the auto-page layout, hopefully will fix cases of automatically placed text overlapping albums. Also improves resizing albums and deals with an edge case of images which load slowly being cropped to the wrong height when added.
Couple fixes for the new thumbnail system, currently when you change thumbnail the change isn't apparent because of the CDN caching but this fix instructs the CDN to purge the thumbnail which has been altered.
Also realised that User profile pics were still hosted on imgur, although they don't seem to have noticed that.  So this will also move them to be KerbalX hosted.

Anyway, I'll deploy those tomorrow after work, there may be a slight downtime or uploading might be disabled for a bit.

 

 

 

[swjr-swis]

22 hours ago, katateochi said:

- If the updated craft has exactly the same name as the one you've already uploaded then just upload it again like normal and you'll be asked if you want to update the existing one or create a new one.  
- If the updated craft has a different name to the one you want to update, then click on your craft and then at the top of the page click "Edit Craft".  Then upload the updated version. You will be asked to confirm that you want to update the current craft.  
Updating a craft will leave the page and pictures etc that you've added as they are, it just updates the file and info about the craft.

I did not know this. I wish I had.

Tip: make an explicit 'Reupload craft' button somewhere. Or make the above explanation easily available somewhere. I bet there's a lot of us that didn't know this was possible.

 

18 hours ago, katateochi said:

I have finally had a reply back from imgur and they are going to lift the block on KerbalX! (that process may take 24 hours).

Good, about time they did their part.

 

Doing good work, @katateochi.

[Candlelight]

I have a problem registering to the site.I got a KerbalX Error 500.

[katateochi]

11 minutes ago, Radon86 said:

I have a problem registering to the site.I got a KerbalX Error 500.

Was that just now? I just deployed some updates to the site so a couple server restarts which would have messed that up for you. Sorry! try again now, should work ok. 

[Candlelight]

1 minute ago, katateochi said:

Was that just now? I just deployed some updates to the site so a couple server restarts which would have messed that up for you. Sorry! try again now, should work ok. 

Thanks! Works perfectly now.

[katateochi]

5 minutes ago, Radon86 said:

Thanks! Works perfectly now.

awesome. Welcome aboard! 

[katateochi]

Site Updates

Over the last week we've had some odd download activity going on, which I assume are bots crawling the site and hitting download urls. So I've changed craft and ckan file download requests from GET requests to POST requests.  This means that they have to originate from the site so something can't just try random craft ids and download craft.  The only difference for users is that links ending in .craft don't work in the same way anymore.  Before you could post a link to a craft with .craft on the end of the url and anyone clicking it would get the craft file directly without being taken to the site.  Now .craft links will take you to the site and then trigger a javascript action to make the download request.  This should cut down on erroneous download activity. 

Let me know if you have any problems downloading craft now.

 

I've added a new set of notifications; you will now get notified if one of your craft is added to a public hangar and when someone favourites your craft (although it doesn't tell you who fav'd).  If you have "open" hangars which other people can add craft to, then you will also get notifications when craft are added to those hangars.  
You can disable these and other notifications under settings https://KerbalX.com/settings?tab=notifications

 

[PeteLeGrand]

I have a problem with my files. 3 of 4 can't be downloaded. Is it because of the filename (spaces and dots) or something else? Username is the same as here.

Thanks a lot in advance!

[katateochi]

1 hour ago, PeteLeGrand said:

I have a problem with my files. 3 of 4 can't be downloaded. Is it because of the filename (spaces and dots) or something else? Username is the same as here.

Thanks a lot in advance!

Yep sorry about that! craft with fullstops in names were not downloadable.  Should be fixed now

 

[PeteLeGrand]

Awesome! Tank you very much...that was quick! 

[Thorman]

uh ok so i have a little issue... actually quite a big issue... so i have this download manager called free download manager and every time i download a craft it shows a notification that shows file not found on server. i also tried to download a craft using google chrome and it shows file not found... ive tried multiple craft... also i tried it a couple days ago thinking it must be kerbal x servers or something but today the same thing happened just now! any ideas what it could be?

[katateochi]

Hi @Thorman, I'm sorry that your download manager isn't working with KX anymore, that is most likely due to recent change I put in place to block bots from downloading craft.  KX was getting crawled by various bots which were downloading craft, this was a) messing with users stats about number of downloads and b) it seemed like someone was trying to hoover up all the craft, for what purpose I don't know, but possibly re-distributing them elsewhere without the user's consent.  

When you tried to DL with chrome was that without your download manager doing it's thing? Downloading as normal from the site by clicking a download link, without any DL tools, should just work as normal. If that's not working can you give me more info (ie tell me the url you are trying to download, info about your browser etc, possibly a screenshot of the link you are trying to click) and I'll look into it.

[Thorman]

3 hours ago, katateochi said:

Hi @Thorman, I'm sorry that your download manager isn't working with KX anymore, that is most likely due to recent change I put in place to block bots from downloading craft.  KX was getting crawled by various bots which were downloading craft, this was a) messing with users stats about number of downloads and b) it seemed like someone was trying to hoover up all the craft, for what purpose I don't know, but possibly re-distributing them elsewhere without the user's consent.  

When you tried to DL with chrome was that without your download manager doing it's thing? Downloading as normal from the site by clicking a download link, without any DL tools, should just work as normal. If that's not working can you give me more info (ie tell me the url you are trying to download, info about your browser etc, possibly a screenshot of the link you are trying to click) and I'll look into it.

Ok so I have the link to the download and yes I did let chrome download by itself but it said some kind of error and immediately stopped downloading... I believe the error was something along the lines of file could not be found... So I clicked more information and chrome sent me to a link that said most likely the file is not available anymore!

heres the link to the craft I'm trying to download

https://kerbalx.com/The_ShadowZone/Duna-Derwisch---Rover-dropship-and-base-delivery