Removal of TXT DBs

[Jonne]

To make things a bit clearer.

Reason for removing TXT support of player database:

• There is individual char_server and char_server_sql, we needed double work for char server edits
  
• Each time a new system appears like mercenaries, we would need to implement both sql and txt saving/loading, increasing our workload
  
• The compiled code is actially different of a SQL and a TXT compiled servers, we need to maintain more project files, people asking for support need to provide additional information etc.
  
• Using SQL is far more flexible because of CP and web services, and should perform better, and be safer ( less likely to loose data ).
  
• They are frequently modified at the runtime.
  
• Maintenance of SQL databases are easier ( deleting old/unused logins, purging invalid data, and etc).
  
• There are TXT related bugs, and unimplemented TXT features such as mercenaries.
  

Now lets look at the db folder:

• db folder does not need maintenance from the server owners.
  
• Supporting db folder in txt format does not increase our workload much, the format is rarely changes, we do not need more project files for that etc.
  
• It is fairly easy to convert some of the dbs to SQL in case of somebody feel more confortable with that
  
• They are not modified on the fly, moreover the server does not modifies it at all.
  
• Does not have bugs or unimplemented features.
  
• Aegis also use flat text files for the same thing.
  

All in all, if somebody fond of SQL db's, he can use it. But dropping TXT support for db folder provides us (developers) only little (or almost none) decrease in our workload, but degrades eA compatibility, makes people harder to switch from eA, and causes additional workload for server owners too ( They now just need to update item_db.txt and they are done. But with SQL dbs, they need to update item_db.sql and import it afterwards, +1 step) . Moreover it makes it harder for those who used to the txt format, to do db work.

One of your reason was, it is easier to edit some dbs in SQL. Well.. if you use a GUI maybe. BUT:

• There are tools available to easily edit TXT format too ( database editors )
  
• If you feel more confortable to edit something in SQL, you are free to do it that way and export afterwards ( i.e. mass edit ).
  

Maybe you feel that db developers would have double work if we maintain 2 dbs. Well, no. They only need to maintain sql, or txt. The one they prefer. And if we provide conversion tools, they can keep updated the other one. ( I think there are some pearl script available to convert dbs, however it would be maybe better to provide binaries, it would be easier to use for all ).

That is my opinion.

Seems quite reasonable. I'd still like seeing the frequently-changing DBs, like item_db, going to SQL. But well, it is everybodies own opinion, since you can change it yourself. Well, either Option 1 or 3, I find Option 2 too much.

[saithis]

With sql DBs it's much easier to integrate the data into websites.

For example a PHP script to view the logs and join them with the killed mob, or the looted item, or a site where the users can view the available ptes and such stuff.

At the moment you need to convert the data to a sql db to use them in a website

[Lemongrass]

So I'll also give my two cents to this topic:

I am totally in favor of dropping TXT support, but as Gepard tries to say, it is really a difficult thing to work with SQL scripts during development. So my suggestion to this is, leave the /db/ and /conf/ folder as it is, but put a shellscript in the root folder that converts these folders to a single or maybe multiple SQL scripts. That would be a lot easier for the developers and also for all users.

As for storing nearly everything into the database like the configs and the static content inside the database folder, I would say it would be no problem, because this data is only requested at server startup. Again I would be in favor of this idea, but not for development only for runtime, because its easier. For those who claim that the descriptions would be lost, it is possible to add a comment field in this configuration database! xD

But all in all I would like to suggest that you maybe also implement SQLite( http://www.sqlite.org/ ), I just looked through the repository's changelog and found an entry for that 7 years ago. It is a local database which would be a perfect replacement for TXT as it can be run locally and also stops the need for TXT support, since it is also an SQL database which uses the same commands as MySQL.

What do you all think of this?

[Arcenciel]

But all in all I would like to suggest that you maybe also implement SQLite( http://www.sqlite.org/ ), I just looked through the repository's changelog and found an entry for that 7 years ago. It is a local database which would be a perfect replacement for TXT as it can be run locally and also stops the need for TXT support, since it is also an SQL database which uses the same commands as MySQL.

What do you all think of this?

There's already a replacement for TXT. It's rAsql.

http://rathena.svn.sourceforge.net/viewvc/rathena/rAsql/

[Lemongrass]

There's already a replacement for TXT. It's rAsql.

http://rathena.svn.s.../rathena/rAsql/

hmm seems like you don't know our TXT users... TXT users are usually the starters that don't know much of anything related to servers/commandshells/etc... so I don't think it will be fitting replacement, since they won't be able to handle this. but that's just my opinion.

off topic:

so if we are allready at removing features and getting the users to the "better way" to do things, wouldn't it be a nice idea to drop the GRF reading support? I didn't really look into it, but does anything use this functionality since the mapcache came up?

[Arcenciel]

hmm seems like you don't know our TXT users... TXT users are usually the starters that don't know much of anything related to servers/commandshells/etc... so I don't think it will be fitting replacement, since they won't be able to handle this. but that's just my opinion.

Doesn't matter. It's already been voted upon that TXT is out of the window. If you looked at rAsql, it doesn't require any set up at all...so what's there to know?

[Lighta]

I don't see why you need tool to edit sql db. Or you count a basic sql client like one but it's quite same as counting notepad for txt... Commad line good enough. About loading I doubt it's longer to start then txt, (except the setup) but if I recall yes some function was less godd written than txt version. Actually that my really CON here, it,s probably gonna be harder to edit those sql function instead txt one since there way much option. Also we already saw some pretty long/lame request in the past. So I wont put too much faith on this.

About configuration file on SQL I don't really see the use since there not so much option but well why not. About commenary they could be moved on doc or wiki so doesn't really matter.

[Hayate Yoshida]

Since TXT is removed as an engine, I think it would be defeating the purpose if we kept hard to edit TXT DBs; besides SQL DBs are MUCH easier to edit and organize. (At least to me they are)

Edited February 9, 2012 by Shinon Yoshida

[ngek202]

can you temporary disable an item thru SQL by adding // (comment)?

[xazax]

No.

[Hayate Yoshida]

Unfortunately no, but I'd rather do a workaround with that than have to try to memorize what every column is. I also like using a central tool. I hate using this tool and that tool for multiple things.

[Gepard]

I also like using a central tool. I hate using this tool and that tool for multiple things.

Me too. This "central tool" is called text editor.

[EvilPuncker]

notepad++ all the way! well, I do like the rAthena SQL idea, but I still think that editing a txt file is way faster/comfortable than editing a sql entry, so my vote goes to 3.

[Hayate Yoshida]

Or MySQL Workbench . It's not central if your DBs are spread out in SQL and TXT. It's only central if it's all the way one direction or all the way the other.

Edited February 10, 2012 by Shinon Yoshida

[OnNplay]

Go for SQL is great. Maybe just for temporary some will feel the pain.

[Hayate Yoshida]

To make things a bit clearer. Reason for removing TXT support of player database:

• There is individual char_server and char_server_sql, we needed double work for char server edits
• Each time a new system appears like mercenaries, we would need to implement both sql and txt saving/loading, increasing our workload
• The compiled code is actially different of a SQL and a TXT compiled servers, we need to maintain more project files, people asking for support need to provide additional information etc.
• Using SQL is far more flexible because of CP and web services, and should perform better, and be safer ( less likely to loose data ).
• They are frequently modified at the runtime.
• Maintenance of SQL databases are easier ( deleting old/unused logins, purging invalid data, and etc).
• There are TXT related bugs, and unimplemented TXT features such as mercenaries.

Now lets look at the db folder:

• db folder does not need maintenance from the server owners.
• Supporting db folder in txt format does not increase our workload much, the format is rarely changes, we do not need more project files for that etc.
• It is fairly easy to convert some of the dbs to SQL in case of somebody feel more confortable with that
• They are not modified on the fly, moreover the server does not modifies it at all.
• Does not have bugs or unimplemented features.
• Aegis also use flat text files for the same thing.

All in all, if somebody fond of SQL db's, he can use it. But dropping TXT support for db folder provides us (developers) only little (or almost none) decrease in our workload, but degrades eA compatibility, makes people harder to switch from eA, and causes additional workload for server owners too ( They now just need to update item_db.txt and they are done. But with SQL dbs, they need to update item_db.sql and import it afterwards, +1 step) . Moreover it makes it harder for those who used to the txt format, to do db work. One of your reason was, it is easier to edit some dbs in SQL. Well.. if you use a GUI maybe. BUT:

• There are tools available to easily edit TXT format too ( database editors )
• If you feel more confortable to edit something in SQL, you are free to do it that way and export afterwards ( i.e. mass edit ).

Maybe you feel that db developers would have double work if we maintain 2 dbs. Well, no. They only need to maintain sql, or txt. The one they prefer. And if we provide conversion tools, they can keep updated the other one. ( I think there are some pearl script available to convert dbs, however it would be maybe better to provide binaries, it would be easier to use for all ). That is my opinion.

Actually I agree with a converter, that would be awesome! I think that would be a great solution!

[Peopleperson49]

I personally like editing TXT documents much much better, however, whatever they choose it all needs to be uniform. Jumping between which files TXT or SQL would be more confusing for some people. Everybody that has created a server has had to learn enough to get it running. SQL is just one more step they have to learn anyway and not that difficult of one. If the choice is txt or SQL then I think we should go only SQL. I have almost 5000 custom items and I can pretty much create an item from my head in TXT without using the help guide. How much harder would it have been to create those same items in SQL? It took no time at all to convert my TXT files to SQL. The fact that SQL loads faster than TXT when you runserver is kind of a mute point, since its minimal and doesn't affect the people that matter (players). I do like how TXT documents are not modified by outside sources preventing unwanted changes or errors, whereas, SQL can be corrupted through general changes, control panel, or other means. I saw first hand when somebody hacked my test server through my control panel and messed up my SQL database. They couldn't touch the TXT files and I was able to convert them back again to fix it. Had I been using TXT all along I would have been safe. Well there's my both sides of the fence speach...

Peopleperson49

Edited March 6, 2012 by peopleperson49

[xazax]

Well.

All source devs agree, there are no technical benefits to porting db folder to SQL. So this is only the matter of taste, which version is fit better to one's workflow. And it looks like the community is split.

[Hayate Yoshida]

Well it seems like the best solution is to create a converter since the community is split.

[Kratos]

Moving all the DB from txt to SQL will take more time for devs to update things and each time they will want to update an item or skill parameter they will need to add an .sql file.

I think most of the devs are comfortable with just replacing the .txt files.

[Nameless2you]

My input to this:

I hate working in sql db, working char logs and stuff is just fine, but item_db logs or mob_db logs in sql are just a horror for me even though it's easy with the name beside it when creating a new one, problem is just it takes so much longer creating it than in txt where you can easily search with a ctrl+f combo, copy paste and then you've got your new item, sql takes you time to select the search button or use the sql tab then type in the stuff, wait for it to load, and then select copy, it's rather annoying when you have large bulks of items you wish to copy/duplicate..

[sketchyphoenix]

So you're saying you are against SQL DBs because whatever front-end you are using is garbage and not anything to do with SQL itself.

[Kratos]

LOL SQL is not a problem at all, but that's up to developers I guess.

@Nameless2you.

As far as editing your own DB is just as easy as doing a select where x = y and doing update x... set x=y where l = m.

I worry more about Script/DB devs needing to add a .SQL file to the emulator.

[ThugLyfe]

Option 3: No, leave them how it is (user selectable between TXT and SQL)

Though SQL IS BETTER!

Edited March 26, 2012 by ThugLyfe

[Hayate Yoshida]

I still believe the best solution would be to create a converter that would convert from TXT to SQL or SQL to TXT.